[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 08:24:53 dist-test-slave-f7th krb5kdc[23481](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 08:24:53 dist-test-slave-f7th krb5kdc[23481](info): set up 2 sockets
Aug 08 08:24:53 dist-test-slave-f7th krb5kdc[23481](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260808 08:24:54.395105 23471 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Aug 08 08:24:54 dist-test-slave-f7th krb5kdc[23481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177494, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T08:24:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:24:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.235.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
I20260808 08:24:54.512759 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:44385
--webserver_interface=127.22.235.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:38081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:44385
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:24:54.622841 23499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:24:54.623095 23499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:24:54.623149 23499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:24:54.627411 23499 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:24:54.627490 23499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:24:54.627523 23499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:24:54.627543 23499 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:24:54.627565 23499 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:24:54.633083 23499 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:38081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:44385
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:44385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.235.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:24:54.634148 23499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:24:54.635164 23499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 08:24:54.643810 23499 server_base.cc:1034] running on GCE node
W20260808 08:24:54.643923 23506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:24:54.643899 23507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:24:54.644228 23509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:24:54.644798 23499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:24:54.645865 23499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:24:54.647018 23499 hybrid_clock.cc:648] HybridClock initialized: now 1786177494646999 us; error 32 us; skew 500 ppm
Aug 08 08:24:54 dist-test-slave-f7th krb5kdc[23481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177494, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:24:54.649830 23499 init.cc:377] Logged in from keytab as kudu/127.22.235.254@KRBTEST.COM (short username kudu)
I20260808 08:24:54.650687 23499 webserver.cc:460] Webserver started at http://127.22.235.254:35729/ using document root <none> and password file <none>
I20260808 08:24:54.651218 23499 fs_manager.cc:359] Metadata directory not provided
I20260808 08:24:54.651275 23499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:24:54.651443 23499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:24:54.653415 23499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "49e77a84c6144fe7881570eb6b70646e"
format_stamp: "Formatted at 2026-08-08 08:24:54 on dist-test-slave-f7th"
server_key: "b086df4f09f03d925a6472e13d8b8d0e"
server_key_iv: "63af943822726a1eafcb9535eae5041e"
server_key_version: "encryptionkey@0"
I20260808 08:24:54.653911 23499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "49e77a84c6144fe7881570eb6b70646e"
format_stamp: "Formatted at 2026-08-08 08:24:54 on dist-test-slave-f7th"
server_key: "b086df4f09f03d925a6472e13d8b8d0e"
server_key_iv: "63af943822726a1eafcb9535eae5041e"
server_key_version: "encryptionkey@0"
I20260808 08:24:54.657516 23499 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 08:24:54.659780 23520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:24:54.660974 23499 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:24:54.661085 23499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "49e77a84c6144fe7881570eb6b70646e"
format_stamp: "Formatted at 2026-08-08 08:24:54 on dist-test-slave-f7th"
server_key: "b086df4f09f03d925a6472e13d8b8d0e"
server_key_iv: "63af943822726a1eafcb9535eae5041e"
server_key_version: "encryptionkey@0"
I20260808 08:24:54.661167 23499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:24:54.690680 23499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:24:54.693765 23499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:24:54.693921 23499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:24:54.700675 23499 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.254:44385
I20260808 08:24:54.700732 23603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.254:44385 every 8 connection(s)
I20260808 08:24:54.701610 23499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:24:54.704396 23604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:24:54.709296 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 23499
I20260808 08:24:54.709463 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:24:54.710292 23604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49e77a84c6144fe7881570eb6b70646e: Bootstrap starting.
I20260808 08:24:54.711120 23471 external_mini_cluster.cc:1428] Setting key 9aacf56523da17b8704e58cb17a1a724
I20260808 08:24:54.713442 23604 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49e77a84c6144fe7881570eb6b70646e: Neither blocks nor log segments found. Creating new log.
I20260808 08:24:54.714286 23604 log.cc:824] T 00000000000000000000000000000000 P 49e77a84c6144fe7881570eb6b70646e: Log is configured to *not* fsync() on all Append() calls
I20260808 08:24:54.716264 23604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49e77a84c6144fe7881570eb6b70646e: No bootstrap required, opened a new log
I20260808 08:24:54.720202 23604 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49e77a84c6144fe7881570eb6b70646e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49e77a84c6144fe7881570eb6b70646e" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 44385 } }
I20260808 08:24:54.720465 23604 raft_consensus.cc:374] T 00000000000000000000000000000000 P 49e77a84c6144fe7881570eb6b70646e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:24:54.720587 23604 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49e77a84c6144fe7881570eb6b70646e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49e77a84c6144fe7881570eb6b70646e, State: Initialized, Role: FOLLOWER
I20260808 08:24:54.721067 23604 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49e77a84c6144fe7881570eb6b70646e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49e77a84c6144fe7881570eb6b70646e" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 44385 } }
I20260808 08:24:54.721195 23604 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49e77a84c6144fe7881570eb6b70646e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:24:54.721257 23604 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49e77a84c6144fe7881570eb6b70646e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:24:54.721350 23604 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49e77a84c6144fe7881570eb6b70646e [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:24:54.722177 23604 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49e77a84c6144fe7881570eb6b70646e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49e77a84c6144fe7881570eb6b70646e" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 44385 } }
I20260808 08:24:54.722486 23604 leader_election.cc:302] T 00000000000000000000000000000000 P 49e77a84c6144fe7881570eb6b70646e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49e77a84c6144fe7881570eb6b70646e; no voters:
I20260808 08:24:54.722784 23604 leader_election.cc:288] T 00000000000000000000000000000000 P 49e77a84c6144fe7881570eb6b70646e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:24:54.722905 23613 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49e77a84c6144fe7881570eb6b70646e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:24:54.723127 23613 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49e77a84c6144fe7881570eb6b70646e [term 1 LEADER]: Becoming Leader. State: Replica: 49e77a84c6144fe7881570eb6b70646e, State: Running, Role: LEADER
I20260808 08:24:54.723435 23613 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49e77a84c6144fe7881570eb6b70646e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49e77a84c6144fe7881570eb6b70646e" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 44385 } }
I20260808 08:24:54.724105 23604 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49e77a84c6144fe7881570eb6b70646e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:24:54.725205 23614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49e77a84c6144fe7881570eb6b70646e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "49e77a84c6144fe7881570eb6b70646e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49e77a84c6144fe7881570eb6b70646e" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 44385 } } }
I20260808 08:24:54.725332 23614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49e77a84c6144fe7881570eb6b70646e [sys.catalog]: This master's current role is: LEADER
I20260808 08:24:54.725632 23615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49e77a84c6144fe7881570eb6b70646e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49e77a84c6144fe7881570eb6b70646e. Latest consensus state: current_term: 1 leader_uuid: "49e77a84c6144fe7881570eb6b70646e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49e77a84c6144fe7881570eb6b70646e" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 44385 } } }
I20260808 08:24:54.725718 23615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49e77a84c6144fe7881570eb6b70646e [sys.catalog]: This master's current role is: LEADER
I20260808 08:24:54.726444 23626 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:24:54.729532 23626 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 08:24:54 dist-test-slave-f7th krb5kdc[23481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177494, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:24:54.735752 23626 catalog_manager.cc:1347] Generated new cluster ID: a32f68f4a2c04a918bdd6bbec47521ad
I20260808 08:24:54.735810 23626 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:24:54.736418 23610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:24:54.714581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52632 (local address 127.22.235.254:44385)
0808 08:24:54.714983 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0808 08:24:54.715000 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 08:24:54.716225 (+ 1225us) server_negotiation.cc:407] Connection header received
0808 08:24:54.719747 (+ 3522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:24:54.719770 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:24:54.720152 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:24:54.720515 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:24:54.722850 (+ 2335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:24:54.724313 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:24:54.726073 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:24:54.726763 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:24:54.726934 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:24:54.730744 (+ 3810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:24:54.730750 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:24:54.730762 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:24:54.730790 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:24:54.732776 (+ 1986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:24:54.733402 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:24:54.733409 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:24:54.733414 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:24:54.733564 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:24:54.733919 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:24:54.733923 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:24:54.733924 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:24:54.734479 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:24:54.734698 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0808 08:24:54.735240 (+ 542us) server_negotiation.cc:299] Negotiation successful
0808 08:24:54.735502 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":105,"threads_started":1}
I20260808 08:24:54.744760 23626 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:24:54.745615 23626 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:24:54.751144 23626 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 49e77a84c6144fe7881570eb6b70646e: Generated new TSK 0
I20260808 08:24:54.751711 23626 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.235.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab.
Entry for principal kudu/127.22.235.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab.
I20260808 08:24:54.783231 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.193:0
--local_ip_for_outbound_sockets=127.22.235.193
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:44385
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab
--principal=kudu/127.22.235.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:38081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:24:54.899526 23642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:24:54.899822 23642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:24:54.899883 23642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:24:54.903918 23642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:24:54.903990 23642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:24:54.904058 23642 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.193
I20260808 08:24:54.909453 23642 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:38081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab
--principal=kudu/127.22.235.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:44385
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23642
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.193
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:24:54.910602 23642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:24:54.911695 23642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:24:54.921600 23655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:24:54.921657 23642 server_base.cc:1034] running on GCE node
W20260808 08:24:54.921615 23649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:24:54.921609 23653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:24:54.922298 23642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:24:54.922935 23642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:24:54.924094 23642 hybrid_clock.cc:648] HybridClock initialized: now 1786177494924072 us; error 32 us; skew 500 ppm
Aug 08 08:24:54 dist-test-slave-f7th krb5kdc[23481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177494, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:24:54.926836 23642 init.cc:377] Logged in from keytab as kudu/127.22.235.193@KRBTEST.COM (short username kudu)
I20260808 08:24:54.927707 23642 webserver.cc:460] Webserver started at http://127.22.235.193:41071/ using document root <none> and password file <none>
I20260808 08:24:54.928244 23642 fs_manager.cc:359] Metadata directory not provided
I20260808 08:24:54.928311 23642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:24:54.928527 23642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:24:54.930598 23642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-0/data/instance:
uuid: "0d119ae4dcdc419889fd80062e1233fc"
format_stamp: "Formatted at 2026-08-08 08:24:54 on dist-test-slave-f7th"
server_key: "36a8c44665d0f1e4625930799ce771ff"
server_key_iv: "a2bd716bb647d10c6ab6a6da895d729d"
server_key_version: "encryptionkey@0"
I20260808 08:24:54.931145 23642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance:
uuid: "0d119ae4dcdc419889fd80062e1233fc"
format_stamp: "Formatted at 2026-08-08 08:24:54 on dist-test-slave-f7th"
server_key: "36a8c44665d0f1e4625930799ce771ff"
server_key_iv: "a2bd716bb647d10c6ab6a6da895d729d"
server_key_version: "encryptionkey@0"
I20260808 08:24:54.934933 23642 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 08:24:54.937417 23667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:24:54.938701 23642 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:24:54.938833 23642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-0/wal
uuid: "0d119ae4dcdc419889fd80062e1233fc"
format_stamp: "Formatted at 2026-08-08 08:24:54 on dist-test-slave-f7th"
server_key: "36a8c44665d0f1e4625930799ce771ff"
server_key_iv: "a2bd716bb647d10c6ab6a6da895d729d"
server_key_version: "encryptionkey@0"
I20260808 08:24:54.938938 23642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:24:54.953372 23642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:24:54.956110 23642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:24:54.956284 23642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:24:54.956943 23642 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:24:54.957783 23642 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:24:54.957839 23642 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:24:54.957893 23642 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:24:54.957919 23642 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:24:54.967024 23642 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.193:39105
I20260808 08:24:54.967051 23837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.193:39105 every 8 connection(s)
I20260808 08:24:54.967940 23642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
I20260808 08:24:54.970033 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 23642
I20260808 08:24:54.970139 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance
I20260808 08:24:54.970438 23471 external_mini_cluster.cc:1428] Setting key 1c82ee6c4ffadbce48731a53b6cd5bd5
Aug 08 08:24:54 dist-test-slave-f7th krb5kdc[23481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177494, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.235.194@KRBTEST.COM; defaulting to no policy
I20260808 08:24:54.982306 23610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:24:54.969832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:43279 (local address 127.22.235.254:44385)
0808 08:24:54.969957 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 08:24:54.969961 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:24:54.970982 (+ 1021us) server_negotiation.cc:407] Connection header received
0808 08:24:54.972474 (+ 1492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:24:54.972477 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:24:54.972537 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:24:54.972607 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:24:54.973780 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:24:54.974432 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:24:54.975840 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:24:54.976183 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:24:54.976288 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:24:54.978923 (+ 2635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:24:54.978927 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:24:54.978930 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:24:54.978960 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:24:54.980435 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:24:54.980939 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:24:54.980943 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:24:54.980945 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:24:54.981052 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:24:54.981398 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:24:54.981401 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:24:54.981402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:24:54.981584 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:24:54.981661 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 08:24:54.982079 (+ 418us) server_negotiation.cc:299] Negotiation successful
0808 08:24:54.982183 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 08:24:54.983117 23841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:24:54.970302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:44385 (local address 127.22.235.193:43279)
0808 08:24:54.970817 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0808 08:24:54.970861 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 08:24:54.972270 (+ 1409us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:24:54.972809 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:24:54.972815 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:24:54.973167 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:24:54.973641 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:24:54.973655 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:24:54.974574 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:24:54.974578 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:24:54.975715 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:24:54.975726 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:24:54.976348 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:24:54.976352 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:24:54.976513 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:24:54.977122 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:24:54.977138 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:24:54.978752 (+ 1614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:24:54.980543 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:24:54.980548 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:24:54.980559 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:24:54.980848 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:24:54.981166 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:24:54.981169 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:24:54.981170 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:24:54.981300 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:24:54.981659 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:24:54.981664 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:24:54.981876 (+ 212us) client_negotiation.cc:769] Sending connection context
0808 08:24:54.982019 (+ 143us) client_negotiation.cc:240] Negotiation successful
0808 08:24:54.982283 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":151,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.194@KRBTEST.COM" created.
I20260808 08:24:54.985222 23838 heartbeater.cc:347] Connected to a master server at 127.22.235.254:44385
I20260808 08:24:54.985481 23838 heartbeater.cc:464] Registering TS with master...
I20260808 08:24:54.986006 23838 heartbeater.cc:511] Master 127.22.235.254:44385 requested a full tablet report, sending...
I20260808 08:24:54.987592 23552 ts_manager.cc:194] Registered new tserver with Master: 0d119ae4dcdc419889fd80062e1233fc (127.22.235.193:39105)
I20260808 08:24:54.988768 23552 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.193@KRBTEST.COM'} at 127.22.235.193:43279
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab.
Entry for principal kudu/127.22.235.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab.
I20260808 08:24:55.000558 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.194:0
--local_ip_for_outbound_sockets=127.22.235.194
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:44385
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab
--principal=kudu/127.22.235.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:38081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:24:55.111830 23851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:24:55.112075 23851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:24:55.112118 23851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:24:55.116025 23851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:24:55.116083 23851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:24:55.116146 23851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.194
I20260808 08:24:55.121237 23851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:38081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab
--principal=kudu/127.22.235.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:44385
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.194
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:24:55.122225 23851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:24:55.123258 23851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:24:55.132709 23866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:24:55.132771 23851 server_base.cc:1034] running on GCE node
W20260808 08:24:55.132795 23858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:24:55.132715 23864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:24:55.133343 23851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:24:55.133929 23851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:24:55.135078 23851 hybrid_clock.cc:648] HybridClock initialized: now 1786177495135062 us; error 28 us; skew 500 ppm
Aug 08 08:24:55 dist-test-slave-f7th krb5kdc[23481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177495, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:24:55.137754 23851 init.cc:377] Logged in from keytab as kudu/127.22.235.194@KRBTEST.COM (short username kudu)
I20260808 08:24:55.138584 23851 webserver.cc:460] Webserver started at http://127.22.235.194:33549/ using document root <none> and password file <none>
I20260808 08:24:55.139118 23851 fs_manager.cc:359] Metadata directory not provided
I20260808 08:24:55.139182 23851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:24:55.139380 23851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:24:55.141368 23851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-1/data/instance:
uuid: "c3bae12cd4e34c178e2c261d610a453c"
format_stamp: "Formatted at 2026-08-08 08:24:55 on dist-test-slave-f7th"
server_key: "24aa123e1df465d29744f8a205e03b2e"
server_key_iv: "67f0f46b6b0bb08359a1fad0447158a0"
server_key_version: "encryptionkey@0"
I20260808 08:24:55.141884 23851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance:
uuid: "c3bae12cd4e34c178e2c261d610a453c"
format_stamp: "Formatted at 2026-08-08 08:24:55 on dist-test-slave-f7th"
server_key: "24aa123e1df465d29744f8a205e03b2e"
server_key_iv: "67f0f46b6b0bb08359a1fad0447158a0"
server_key_version: "encryptionkey@0"
I20260808 08:24:55.145470 23851 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 08:24:55.147717 23875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:24:55.148909 23851 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:24:55.149017 23851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-1/wal
uuid: "c3bae12cd4e34c178e2c261d610a453c"
format_stamp: "Formatted at 2026-08-08 08:24:55 on dist-test-slave-f7th"
server_key: "24aa123e1df465d29744f8a205e03b2e"
server_key_iv: "67f0f46b6b0bb08359a1fad0447158a0"
server_key_version: "encryptionkey@0"
I20260808 08:24:55.149107 23851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:24:55.175568 23851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:24:55.178316 23851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:24:55.178495 23851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:24:55.179081 23851 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:24:55.179910 23851 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:24:55.179958 23851 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:24:55.180022 23851 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:24:55.180048 23851 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:24:55.188719 23851 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.194:36019
I20260808 08:24:55.188753 24035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.194:36019 every 8 connection(s)
I20260808 08:24:55.189571 23851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
I20260808 08:24:55.197124 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 23851
I20260808 08:24:55.197245 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance
I20260808 08:24:55.197559 23471 external_mini_cluster.cc:1428] Setting key 0e80381437de4ff8bd6ed2882fca1104
Aug 08 08:24:55 dist-test-slave-f7th krb5kdc[23481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177495, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.194@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:24:55.202607 23610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:24:55.191377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:53219 (local address 127.22.235.254:44385)
0808 08:24:55.191553 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 08:24:55.191557 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:24:55.192387 (+ 830us) server_negotiation.cc:407] Connection header received
0808 08:24:55.193666 (+ 1279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:24:55.193669 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:24:55.193715 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:24:55.193777 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:24:55.194864 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:24:55.195376 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:24:55.196525 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:24:55.196785 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:24:55.196865 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:24:55.199369 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:24:55.199373 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:24:55.199375 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:24:55.199398 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:24:55.200728 (+ 1330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:24:55.201193 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:24:55.201197 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:24:55.201199 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:24:55.201275 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:24:55.201576 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:24:55.201580 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:24:55.201581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:24:55.201757 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:24:55.201843 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 08:24:55.202317 (+ 474us) server_negotiation.cc:299] Negotiation successful
0808 08:24:55.202441 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 08:24:55.203359 24038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:24:55.191755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:44385 (local address 127.22.235.194:53219)
0808 08:24:55.192233 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0808 08:24:55.192276 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 08:24:55.193459 (+ 1183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:24:55.193916 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:24:55.193922 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:24:55.194281 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:24:55.194746 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:24:55.194759 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:24:55.195481 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:24:55.195484 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:24:55.196424 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:24:55.196435 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:24:55.196950 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:24:55.196954 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:24:55.197142 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:24:55.197714 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:24:55.197734 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:24:55.199218 (+ 1484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:24:55.200836 (+ 1618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:24:55.200842 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:24:55.200857 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:24:55.201103 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:24:55.201354 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:24:55.201357 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:24:55.201359 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:24:55.201489 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:24:55.201853 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:24:55.201859 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:24:55.202097 (+ 238us) client_negotiation.cc:769] Sending connection context
0808 08:24:55.202257 (+ 160us) client_negotiation.cc:240] Negotiation successful
0808 08:24:55.202559 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":118,"threads_started":1}
I20260808 08:24:55.204443 24036 heartbeater.cc:347] Connected to a master server at 127.22.235.254:44385
I20260808 08:24:55.204694 24036 heartbeater.cc:464] Registering TS with master...
I20260808 08:24:55.205158 24036 heartbeater.cc:511] Master 127.22.235.254:44385 requested a full tablet report, sending...
I20260808 08:24:55.206174 23552 ts_manager.cc:194] Registered new tserver with Master: c3bae12cd4e34c178e2c261d610a453c (127.22.235.194:36019)
I20260808 08:24:55.206969 23552 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.194@KRBTEST.COM'} at 127.22.235.194:53219
WARNING: no policy specified for kudu/127.22.235.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab.
Entry for principal kudu/127.22.235.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab.
I20260808 08:24:55.223683 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.195:0
--local_ip_for_outbound_sockets=127.22.235.195
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:44385
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab
--principal=kudu/127.22.235.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:38081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:24:55.337440 24043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:24:55.337729 24043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:24:55.337800 24043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:24:55.341811 24043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:24:55.341863 24043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:24:55.341933 24043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.195
I20260808 08:24:55.347100 24043 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:38081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab
--principal=kudu/127.22.235.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:44385
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.195
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:24:55.348078 24043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:24:55.349093 24043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:24:55.358289 24051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:24:55.358369 24050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:24:55.358325 24043 server_base.cc:1034] running on GCE node
W20260808 08:24:55.358310 24053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:24:55.358916 24043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:24:55.359539 24043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:24:55.360682 24043 hybrid_clock.cc:648] HybridClock initialized: now 1786177495360664 us; error 27 us; skew 500 ppm
Aug 08 08:24:55 dist-test-slave-f7th krb5kdc[23481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177495, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:24:55.363276 24043 init.cc:377] Logged in from keytab as kudu/127.22.235.195@KRBTEST.COM (short username kudu)
I20260808 08:24:55.364179 24043 webserver.cc:460] Webserver started at http://127.22.235.195:40989/ using document root <none> and password file <none>
I20260808 08:24:55.364758 24043 fs_manager.cc:359] Metadata directory not provided
I20260808 08:24:55.364823 24043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:24:55.365002 24043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:24:55.366981 24043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-2/data/instance:
uuid: "7c1c879e1b2a4bbc9a5840a440c7e01a"
format_stamp: "Formatted at 2026-08-08 08:24:55 on dist-test-slave-f7th"
server_key: "9b13b189ac2c32b7d6098fc34ae072bb"
server_key_iv: "ad451f2dabc9a0732c4079a82e32d051"
server_key_version: "encryptionkey@0"
I20260808 08:24:55.367480 24043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance:
uuid: "7c1c879e1b2a4bbc9a5840a440c7e01a"
format_stamp: "Formatted at 2026-08-08 08:24:55 on dist-test-slave-f7th"
server_key: "9b13b189ac2c32b7d6098fc34ae072bb"
server_key_iv: "ad451f2dabc9a0732c4079a82e32d051"
server_key_version: "encryptionkey@0"
I20260808 08:24:55.371006 24043 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 08:24:55.373355 24064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:24:55.374536 24043 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:24:55.374643 24043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-2/wal
uuid: "7c1c879e1b2a4bbc9a5840a440c7e01a"
format_stamp: "Formatted at 2026-08-08 08:24:55 on dist-test-slave-f7th"
server_key: "9b13b189ac2c32b7d6098fc34ae072bb"
server_key_iv: "ad451f2dabc9a0732c4079a82e32d051"
server_key_version: "encryptionkey@0"
I20260808 08:24:55.374737 24043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:24:55.388463 24043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:24:55.391083 24043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:24:55.391255 24043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:24:55.391811 24043 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:24:55.392659 24043 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:24:55.392707 24043 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:24:55.392761 24043 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:24:55.392791 24043 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:24:55.401309 24043 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.195:35415
I20260808 08:24:55.401331 24237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.195:35415 every 8 connection(s)
I20260808 08:24:55.402166 24043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
I20260808 08:24:55.409650 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 24043
I20260808 08:24:55.409762 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance
I20260808 08:24:55.410027 23471 external_mini_cluster.cc:1428] Setting key b1399ba38606189dfc23a5e960ca5891
Aug 08 08:24:55 dist-test-slave-f7th krb5kdc[23481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177495, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.195@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:24:55.415200 23610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:24:55.404013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:37295 (local address 127.22.235.254:44385)
0808 08:24:55.404163 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 08:24:55.404168 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:24:55.405004 (+ 836us) server_negotiation.cc:407] Connection header received
0808 08:24:55.406212 (+ 1208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:24:55.406215 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:24:55.406263 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:24:55.406334 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:24:55.407433 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:24:55.407947 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:24:55.409035 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:24:55.409247 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:24:55.409319 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:24:55.412022 (+ 2703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:24:55.412025 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:24:55.412028 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:24:55.412052 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:24:55.413456 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:24:55.413904 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:24:55.413906 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:24:55.413907 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:24:55.413973 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:24:55.414267 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:24:55.414271 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:24:55.414272 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:24:55.414475 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:24:55.414564 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 08:24:55.414983 (+ 419us) server_negotiation.cc:299] Negotiation successful
0808 08:24:55.415082 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 08:24:55.416018 24242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:24:55.404400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:44385 (local address 127.22.235.195:37295)
0808 08:24:55.404855 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0808 08:24:55.404896 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 08:24:55.406026 (+ 1130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:24:55.406481 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:24:55.406488 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:24:55.406839 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:24:55.407321 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:24:55.407335 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:24:55.408044 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:24:55.408046 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:24:55.408934 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:24:55.408943 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:24:55.409349 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:24:55.409352 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:24:55.409522 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:24:55.410122 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:24:55.410142 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:24:55.411882 (+ 1740us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:24:55.413574 (+ 1692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:24:55.413578 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:24:55.413590 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:24:55.413828 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:24:55.414033 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:24:55.414035 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:24:55.414037 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:24:55.414156 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:24:55.414560 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:24:55.414564 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:24:55.414778 (+ 214us) client_negotiation.cc:769] Sending connection context
0808 08:24:55.414925 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 08:24:55.415174 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":137,"threads_started":1}
I20260808 08:24:55.417171 24238 heartbeater.cc:347] Connected to a master server at 127.22.235.254:44385
I20260808 08:24:55.417392 24238 heartbeater.cc:464] Registering TS with master...
I20260808 08:24:55.417850 24238 heartbeater.cc:511] Master 127.22.235.254:44385 requested a full tablet report, sending...
I20260808 08:24:55.418809 23552 ts_manager.cc:194] Registered new tserver with Master: 7c1c879e1b2a4bbc9a5840a440c7e01a (127.22.235.195:35415)
I20260808 08:24:55.419348 23552 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.195@KRBTEST.COM'} at 127.22.235.195:37295
I20260808 08:24:55.424283 23471 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 08:24:55 dist-test-slave-f7th krb5kdc[23481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177495, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 08:24:55 dist-test-slave-f7th krb5kdc[23481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177495, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.235.193@KRBTEST.COM
I20260808 08:24:55.450301 24248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:24:55.441623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54028 (local address 127.22.235.193:39105)
0808 08:24:55.441909 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0808 08:24:55.441923 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 08:24:55.441949 (+ 26us) server_negotiation.cc:407] Connection header received
0808 08:24:55.442041 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:24:55.442047 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:24:55.442235 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:24:55.442360 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:24:55.442983 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:24:55.443778 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:24:55.444573 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:24:55.444982 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:24:55.445064 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:24:55.447274 (+ 2210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:24:55.447284 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:24:55.447297 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:24:55.447322 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:24:55.448741 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:24:55.449148 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:24:55.449154 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:24:55.449155 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:24:55.449237 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:24:55.449423 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:24:55.449425 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:24:55.449426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:24:55.449651 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:24:55.449822 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0808 08:24:55.449944 (+ 122us) server_negotiation.cc:299] Negotiation successful
0808 08:24:55.450124 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":72,"threads_started":1}
W20260808 08:24:55.451259 23711 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54028
Aug 08 08:24:55 dist-test-slave-f7th krb5kdc[23481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177495, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 08:24:55 dist-test-slave-f7th krb5kdc[23481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177495, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.235.193@KRBTEST.COM
I20260808 08:24:55.477138 24248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:24:55.469718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54042 (local address 127.22.235.193:39105)
0808 08:24:55.469865 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 08:24:55.469870 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:24:55.469998 (+ 128us) server_negotiation.cc:407] Connection header received
0808 08:24:55.470139 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:24:55.470142 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:24:55.470183 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:24:55.470252 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:24:55.470943 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:24:55.471460 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:24:55.472264 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:24:55.472494 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:24:55.472582 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:24:55.474735 (+ 2153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:24:55.474738 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:24:55.474740 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:24:55.474763 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:24:55.475845 (+ 1082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:24:55.476195 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:24:55.476197 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:24:55.476198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:24:55.476266 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:24:55.476443 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:24:55.476445 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:24:55.476446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:24:55.476593 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:24:55.476733 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 08:24:55.476905 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 08:24:55.477013 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 08:24:55.478111 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 23642
I20260808 08:24:55.497509 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 23851
I20260808 08:24:55.506295 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 24043
I20260808 08:24:55.520463 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 23499
2026-08-08T08:24:55Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2313 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 08:24:55 dist-test-slave-f7th krb5kdc[24259](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 08:24:55 dist-test-slave-f7th krb5kdc[24259](info): set up 2 sockets
Aug 08 08:24:55 dist-test-slave-f7th krb5kdc[24259](info): commencing operation
krb5kdc: starting...
W20260808 08:24:56.678750 23471 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Aug 08 08:24:56 dist-test-slave-f7th krb5kdc[24259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T08:24:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:24:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.235.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
I20260808 08:24:56.793330 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:36873
--webserver_interface=127.22.235.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:39003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:36873
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:24:56.904562 24283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:24:56.904876 24283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:24:56.904935 24283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:24:56.908932 24283 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:24:56.908985 24283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:24:56.909004 24283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:24:56.909020 24283 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:24:56.909037 24283 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:24:56.914142 24283 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:39003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:36873
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:36873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.235.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:24:56.915071 24283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:24:56.916000 24283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:24:56.924172 24292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:24:56.924243 24283 server_base.cc:1034] running on GCE node
W20260808 08:24:56.924152 24291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:24:56.924139 24295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:24:56.924997 24283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:24:56.926048 24283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:24:56.927210 24283 hybrid_clock.cc:648] HybridClock initialized: now 1786177496927190 us; error 30 us; skew 500 ppm
Aug 08 08:24:56 dist-test-slave-f7th krb5kdc[24259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177496, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:24:56.929891 24283 init.cc:377] Logged in from keytab as kudu/127.22.235.254@KRBTEST.COM (short username kudu)
I20260808 08:24:56.930745 24283 webserver.cc:460] Webserver started at http://127.22.235.254:40339/ using document root <none> and password file <none>
I20260808 08:24:56.931288 24283 fs_manager.cc:359] Metadata directory not provided
I20260808 08:24:56.931353 24283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:24:56.931561 24283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:24:56.933612 24283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "5da5d4d45c0549c7b059ae3d6d98ea9e"
format_stamp: "Formatted at 2026-08-08 08:24:56 on dist-test-slave-f7th"
server_key: "1999f1ecc923ba21dd950020ef6137f0"
server_key_iv: "f5eb91af27423eb58a112ce658660b97"
server_key_version: "encryptionkey@0"
I20260808 08:24:56.934139 24283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "5da5d4d45c0549c7b059ae3d6d98ea9e"
format_stamp: "Formatted at 2026-08-08 08:24:56 on dist-test-slave-f7th"
server_key: "1999f1ecc923ba21dd950020ef6137f0"
server_key_iv: "f5eb91af27423eb58a112ce658660b97"
server_key_version: "encryptionkey@0"
I20260808 08:24:56.937690 24283 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260808 08:24:56.940045 24303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:24:56.941252 24283 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:24:56.941365 24283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "5da5d4d45c0549c7b059ae3d6d98ea9e"
format_stamp: "Formatted at 2026-08-08 08:24:56 on dist-test-slave-f7th"
server_key: "1999f1ecc923ba21dd950020ef6137f0"
server_key_iv: "f5eb91af27423eb58a112ce658660b97"
server_key_version: "encryptionkey@0"
I20260808 08:24:56.941460 24283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:24:56.953773 24283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:24:56.956334 24283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:24:56.956498 24283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:24:56.963469 24283 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.254:36873
I20260808 08:24:56.963486 24384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.254:36873 every 8 connection(s)
I20260808 08:24:56.964342 24283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:24:56.967172 24386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:24:56.968942 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 24283
I20260808 08:24:56.969046 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:24:56.969293 23471 external_mini_cluster.cc:1428] Setting key 33b3dbc6e309900bf7bf2a0ac54b1dda
I20260808 08:24:56.973362 24386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5da5d4d45c0549c7b059ae3d6d98ea9e: Bootstrap starting.
I20260808 08:24:56.975560 24386 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5da5d4d45c0549c7b059ae3d6d98ea9e: Neither blocks nor log segments found. Creating new log.
I20260808 08:24:56.976374 24386 log.cc:824] T 00000000000000000000000000000000 P 5da5d4d45c0549c7b059ae3d6d98ea9e: Log is configured to *not* fsync() on all Append() calls
Aug 08 08:24:56 dist-test-slave-f7th krb5kdc[24259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:24:56.978282 24386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5da5d4d45c0549c7b059ae3d6d98ea9e: No bootstrap required, opened a new log
I20260808 08:24:56.982158 24392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:24:56.970683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36430 (local address 127.22.235.254:36873)
0808 08:24:56.971105 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0808 08:24:56.971120 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 08:24:56.971155 (+ 35us) server_negotiation.cc:407] Connection header received
0808 08:24:56.971786 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:24:56.971803 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:24:56.972124 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:24:56.972418 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:24:56.973206 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:24:56.974178 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:24:56.974980 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:24:56.975427 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:24:56.975582 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:24:56.977900 (+ 2318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:24:56.977905 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:24:56.977919 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:24:56.977945 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:24:56.979501 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:24:56.979981 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:24:56.979985 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:24:56.979989 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:24:56.980106 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:24:56.980373 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:24:56.980375 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:24:56.980376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:24:56.980767 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:24:56.980914 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 08:24:56.981182 (+ 268us) server_negotiation.cc:299] Negotiation successful
0808 08:24:56.981442 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":137,"threads_started":1}
I20260808 08:24:56.982190 24386 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5da5d4d45c0549c7b059ae3d6d98ea9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5da5d4d45c0549c7b059ae3d6d98ea9e" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 36873 } }
I20260808 08:24:56.982446 24386 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5da5d4d45c0549c7b059ae3d6d98ea9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:24:56.982515 24386 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5da5d4d45c0549c7b059ae3d6d98ea9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5da5d4d45c0549c7b059ae3d6d98ea9e, State: Initialized, Role: FOLLOWER
I20260808 08:24:56.982947 24386 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5da5d4d45c0549c7b059ae3d6d98ea9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5da5d4d45c0549c7b059ae3d6d98ea9e" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 36873 } }
I20260808 08:24:56.983095 24386 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5da5d4d45c0549c7b059ae3d6d98ea9e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:24:56.983162 24386 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5da5d4d45c0549c7b059ae3d6d98ea9e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:24:56.983263 24386 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5da5d4d45c0549c7b059ae3d6d98ea9e [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:24:56.984302 24386 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5da5d4d45c0549c7b059ae3d6d98ea9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5da5d4d45c0549c7b059ae3d6d98ea9e" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 36873 } }
I20260808 08:24:56.984664 24386 leader_election.cc:302] T 00000000000000000000000000000000 P 5da5d4d45c0549c7b059ae3d6d98ea9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5da5d4d45c0549c7b059ae3d6d98ea9e; no voters:
I20260808 08:24:56.984987 24386 leader_election.cc:288] T 00000000000000000000000000000000 P 5da5d4d45c0549c7b059ae3d6d98ea9e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:24:56.985064 24394 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5da5d4d45c0549c7b059ae3d6d98ea9e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:24:56.985265 24394 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5da5d4d45c0549c7b059ae3d6d98ea9e [term 1 LEADER]: Becoming Leader. State: Replica: 5da5d4d45c0549c7b059ae3d6d98ea9e, State: Running, Role: LEADER
I20260808 08:24:56.985562 24394 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5da5d4d45c0549c7b059ae3d6d98ea9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5da5d4d45c0549c7b059ae3d6d98ea9e" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 36873 } }
I20260808 08:24:56.986171 24386 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5da5d4d45c0549c7b059ae3d6d98ea9e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:24:56.987214 24396 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5da5d4d45c0549c7b059ae3d6d98ea9e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5da5d4d45c0549c7b059ae3d6d98ea9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5da5d4d45c0549c7b059ae3d6d98ea9e" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 36873 } } }
I20260808 08:24:56.987344 24396 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5da5d4d45c0549c7b059ae3d6d98ea9e [sys.catalog]: This master's current role is: LEADER
I20260808 08:24:56.987622 24398 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5da5d4d45c0549c7b059ae3d6d98ea9e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5da5d4d45c0549c7b059ae3d6d98ea9e. Latest consensus state: current_term: 1 leader_uuid: "5da5d4d45c0549c7b059ae3d6d98ea9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5da5d4d45c0549c7b059ae3d6d98ea9e" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 36873 } } }
I20260808 08:24:56.987715 24398 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5da5d4d45c0549c7b059ae3d6d98ea9e [sys.catalog]: This master's current role is: LEADER
I20260808 08:24:56.987818 24410 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:24:56.990899 24410 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:24:56.996027 24410 catalog_manager.cc:1347] Generated new cluster ID: 35c416f82be54297aace3b3e388a056e
I20260808 08:24:56.996090 24410 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:24:57.016912 24410 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:24:57.017866 24410 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:24:57.021296 24410 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5da5d4d45c0549c7b059ae3d6d98ea9e: Generated new TSK 0
I20260808 08:24:57.022310 24410 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.235.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab.
Entry for principal kudu/127.22.235.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab.
I20260808 08:24:57.059331 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.193:0
--local_ip_for_outbound_sockets=127.22.235.193
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:36873
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab
--principal=kudu/127.22.235.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:39003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:24:57.179903 24427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:24:57.180138 24427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:24:57.180178 24427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:24:57.184073 24427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:24:57.184128 24427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:24:57.184181 24427 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.193
I20260808 08:24:57.189235 24427 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:39003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab
--principal=kudu/127.22.235.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:36873
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.193
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:24:57.190199 24427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:24:57.191288 24427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:24:57.200301 24433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:24:57.200327 24434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:24:57.200304 24437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:24:57.200701 24427 server_base.cc:1034] running on GCE node
I20260808 08:24:57.201041 24427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:24:57.201601 24427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:24:57.202750 24427 hybrid_clock.cc:648] HybridClock initialized: now 1786177497202734 us; error 27 us; skew 500 ppm
Aug 08 08:24:57 dist-test-slave-f7th krb5kdc[24259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177497, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:24:57.205406 24427 init.cc:377] Logged in from keytab as kudu/127.22.235.193@KRBTEST.COM (short username kudu)
I20260808 08:24:57.206183 24427 webserver.cc:460] Webserver started at http://127.22.235.193:34413/ using document root <none> and password file <none>
I20260808 08:24:57.206715 24427 fs_manager.cc:359] Metadata directory not provided
I20260808 08:24:57.206779 24427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:24:57.206969 24427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:24:57.208932 24427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-0/data/instance:
uuid: "0be8f012a1574e399bcbadbebdec07a5"
format_stamp: "Formatted at 2026-08-08 08:24:57 on dist-test-slave-f7th"
server_key: "7eccc6ef4fe5c4ea105909de96fe1d96"
server_key_iv: "ec15f3c7769fb58725c8b3e939f12788"
server_key_version: "encryptionkey@0"
I20260808 08:24:57.209421 24427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance:
uuid: "0be8f012a1574e399bcbadbebdec07a5"
format_stamp: "Formatted at 2026-08-08 08:24:57 on dist-test-slave-f7th"
server_key: "7eccc6ef4fe5c4ea105909de96fe1d96"
server_key_iv: "ec15f3c7769fb58725c8b3e939f12788"
server_key_version: "encryptionkey@0"
I20260808 08:24:57.212812 24427 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 08:24:57.215143 24446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:24:57.216295 24427 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:24:57.216392 24427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-0/wal
uuid: "0be8f012a1574e399bcbadbebdec07a5"
format_stamp: "Formatted at 2026-08-08 08:24:57 on dist-test-slave-f7th"
server_key: "7eccc6ef4fe5c4ea105909de96fe1d96"
server_key_iv: "ec15f3c7769fb58725c8b3e939f12788"
server_key_version: "encryptionkey@0"
I20260808 08:24:57.216485 24427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:24:57.239548 24427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:24:57.242264 24427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:24:57.242416 24427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:24:57.242951 24427 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:24:57.243747 24427 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:24:57.243794 24427 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:24:57.243834 24427 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:24:57.243853 24427 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:24:57.255062 24427 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.193:36805
I20260808 08:24:57.255134 24623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.193:36805 every 8 connection(s)
I20260808 08:24:57.256039 24427 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
Aug 08 08:24:57 dist-test-slave-f7th krb5kdc[24259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177497, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:24:57.265998 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 24427
I20260808 08:24:57.266124 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance
I20260808 08:24:57.266410 23471 external_mini_cluster.cc:1428] Setting key 54e6ecc565cfeec03a7323f4bcd437bc
I20260808 08:24:57.268985 24392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:24:57.257837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:56913 (local address 127.22.235.254:36873)
0808 08:24:57.257975 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 08:24:57.257980 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:24:57.258832 (+ 852us) server_negotiation.cc:407] Connection header received
0808 08:24:57.260021 (+ 1189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:24:57.260024 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:24:57.260073 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:24:57.260147 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:24:57.261312 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:24:57.261838 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:24:57.262878 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:24:57.263094 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:24:57.263163 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:24:57.265455 (+ 2292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:24:57.265458 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:24:57.265460 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:24:57.265492 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:24:57.266950 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:24:57.267413 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:24:57.267417 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:24:57.267419 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:24:57.267504 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:24:57.267854 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:24:57.267858 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:24:57.267859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:24:57.268080 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:24:57.268161 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 08:24:57.268755 (+ 594us) server_negotiation.cc:299] Negotiation successful
0808 08:24:57.268863 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 08:24:57.270054 24627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:24:57.258246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:36873 (local address 127.22.235.193:56913)
0808 08:24:57.258688 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0808 08:24:57.258728 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 08:24:57.259832 (+ 1104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:24:57.260295 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:24:57.260301 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:24:57.260701 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:24:57.261184 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:24:57.261197 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:24:57.261929 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:24:57.261931 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:24:57.262787 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:24:57.262796 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:24:57.263197 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:24:57.263199 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:24:57.263333 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:24:57.263878 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:24:57.263894 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:24:57.265292 (+ 1398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:24:57.267057 (+ 1765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:24:57.267063 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:24:57.267078 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:24:57.267319 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:24:57.267589 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:24:57.267592 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:24:57.267594 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:24:57.267738 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:24:57.268188 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:24:57.268195 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:24:57.268494 (+ 299us) client_negotiation.cc:769] Sending connection context
0808 08:24:57.268729 (+ 235us) client_negotiation.cc:240] Negotiation successful
0808 08:24:57.269062 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":110,"threads_started":1}
I20260808 08:24:57.271195 24624 heartbeater.cc:347] Connected to a master server at 127.22.235.254:36873
I20260808 08:24:57.271436 24624 heartbeater.cc:464] Registering TS with master...
I20260808 08:24:57.271974 24624 heartbeater.cc:511] Master 127.22.235.254:36873 requested a full tablet report, sending...
I20260808 08:24:57.273557 24328 ts_manager.cc:194] Registered new tserver with Master: 0be8f012a1574e399bcbadbebdec07a5 (127.22.235.193:36805)
I20260808 08:24:57.274665 24328 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.193@KRBTEST.COM'} at 127.22.235.193:56913
WARNING: no policy specified for kudu/127.22.235.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab.
Entry for principal kudu/127.22.235.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab.
I20260808 08:24:57.293342 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.194:0
--local_ip_for_outbound_sockets=127.22.235.194
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:36873
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab
--principal=kudu/127.22.235.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:39003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:24:57.404191 24631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:24:57.404453 24631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:24:57.404516 24631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:24:57.408526 24631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:24:57.408586 24631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:24:57.408684 24631 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.194
I20260808 08:24:57.413789 24631 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:39003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab
--principal=kudu/127.22.235.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:36873
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24631
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.194
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:24:57.414801 24631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:24:57.415786 24631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:24:57.425046 24639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:24:57.425124 24631 server_base.cc:1034] running on GCE node
W20260808 08:24:57.425132 24636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:24:57.425132 24637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:24:57.425660 24631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:24:57.426285 24631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:24:57.427441 24631 hybrid_clock.cc:648] HybridClock initialized: now 1786177497427426 us; error 27 us; skew 500 ppm
Aug 08 08:24:57 dist-test-slave-f7th krb5kdc[24259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177497, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:24:57.430088 24631 init.cc:377] Logged in from keytab as kudu/127.22.235.194@KRBTEST.COM (short username kudu)
I20260808 08:24:57.430908 24631 webserver.cc:460] Webserver started at http://127.22.235.194:44225/ using document root <none> and password file <none>
I20260808 08:24:57.431438 24631 fs_manager.cc:359] Metadata directory not provided
I20260808 08:24:57.431507 24631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:24:57.431720 24631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:24:57.433840 24631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-1/data/instance:
uuid: "10a548dd59364a70a3246b6e14f7914b"
format_stamp: "Formatted at 2026-08-08 08:24:57 on dist-test-slave-f7th"
server_key: "23ddab7c1b8f8a546ff584140fa60c30"
server_key_iv: "8d04122585fd3f1404ccd60d283f6a1d"
server_key_version: "encryptionkey@0"
I20260808 08:24:57.434362 24631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance:
uuid: "10a548dd59364a70a3246b6e14f7914b"
format_stamp: "Formatted at 2026-08-08 08:24:57 on dist-test-slave-f7th"
server_key: "23ddab7c1b8f8a546ff584140fa60c30"
server_key_iv: "8d04122585fd3f1404ccd60d283f6a1d"
server_key_version: "encryptionkey@0"
I20260808 08:24:57.437914 24631 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 08:24:57.440234 24651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:24:57.441488 24631 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:24:57.441591 24631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-1/wal
uuid: "10a548dd59364a70a3246b6e14f7914b"
format_stamp: "Formatted at 2026-08-08 08:24:57 on dist-test-slave-f7th"
server_key: "23ddab7c1b8f8a546ff584140fa60c30"
server_key_iv: "8d04122585fd3f1404ccd60d283f6a1d"
server_key_version: "encryptionkey@0"
I20260808 08:24:57.441684 24631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:24:57.458454 24631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:24:57.461138 24631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:24:57.461299 24631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:24:57.461848 24631 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:24:57.462653 24631 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:24:57.462699 24631 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:24:57.462739 24631 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:24:57.462767 24631 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:24:57.471549 24631 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.194:44851
I20260808 08:24:57.471588 24840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.194:44851 every 8 connection(s)
I20260808 08:24:57.472398 24631 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
I20260808 08:24:57.479573 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 24631
I20260808 08:24:57.479697 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance
I20260808 08:24:57.479950 23471 external_mini_cluster.cc:1428] Setting key 09f7815631a5a07e45dfae3e258c261a
Aug 08 08:24:57 dist-test-slave-f7th krb5kdc[24259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177497, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.194@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:24:57.485337 24392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:24:57.474120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:41737 (local address 127.22.235.254:36873)
0808 08:24:57.474257 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 08:24:57.474262 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:24:57.475130 (+ 868us) server_negotiation.cc:407] Connection header received
0808 08:24:57.476300 (+ 1170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:24:57.476304 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:24:57.476355 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:24:57.476429 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:24:57.477523 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:24:57.478110 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:24:57.479192 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:24:57.479490 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:24:57.479608 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:24:57.482068 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:24:57.482071 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:24:57.482073 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:24:57.482096 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:24:57.483392 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:24:57.483865 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:24:57.483869 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:24:57.483870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:24:57.483944 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:24:57.484259 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:24:57.484262 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:24:57.484264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:24:57.484437 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:24:57.484525 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 08:24:57.485050 (+ 525us) server_negotiation.cc:299] Negotiation successful
0808 08:24:57.485174 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 08:24:57.486189 24843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:24:57.474538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:36873 (local address 127.22.235.194:41737)
0808 08:24:57.474980 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0808 08:24:57.475028 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 08:24:57.476108 (+ 1080us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:24:57.476571 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:24:57.476578 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:24:57.476932 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:24:57.477394 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:24:57.477407 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:24:57.478212 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:24:57.478215 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:24:57.479071 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:24:57.479080 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:24:57.479621 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:24:57.479623 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:24:57.479796 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:24:57.480313 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:24:57.480328 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:24:57.481911 (+ 1583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:24:57.483510 (+ 1599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:24:57.483515 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:24:57.483531 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:24:57.483771 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:24:57.484034 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:24:57.484037 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:24:57.484039 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:24:57.484172 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:24:57.484538 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:24:57.484544 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:24:57.484820 (+ 276us) client_negotiation.cc:769] Sending connection context
0808 08:24:57.484989 (+ 169us) client_negotiation.cc:240] Negotiation successful
0808 08:24:57.485294 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":104,"threads_started":1}
I20260808 08:24:57.487356 24841 heartbeater.cc:347] Connected to a master server at 127.22.235.254:36873
I20260808 08:24:57.487613 24841 heartbeater.cc:464] Registering TS with master...
I20260808 08:24:57.488193 24841 heartbeater.cc:511] Master 127.22.235.254:36873 requested a full tablet report, sending...
I20260808 08:24:57.489356 24328 ts_manager.cc:194] Registered new tserver with Master: 10a548dd59364a70a3246b6e14f7914b (127.22.235.194:44851)
I20260808 08:24:57.490127 24328 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.194@KRBTEST.COM'} at 127.22.235.194:41737
WARNING: no policy specified for kudu/127.22.235.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab.
Entry for principal kudu/127.22.235.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab.
I20260808 08:24:57.505285 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.195:0
--local_ip_for_outbound_sockets=127.22.235.195
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:36873
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab
--principal=kudu/127.22.235.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:39003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:24:57.618754 24848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:24:57.619107 24848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:24:57.619159 24848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:24:57.623358 24848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:24:57.623420 24848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:24:57.623477 24848 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.195
I20260808 08:24:57.629195 24848 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:39003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab
--principal=kudu/127.22.235.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:36873
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24848
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.195
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:24:57.630262 24848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:24:57.631285 24848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:24:57.640867 24856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:24:57.640867 24854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:24:57.640878 24853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:24:57.641112 24848 server_base.cc:1034] running on GCE node
I20260808 08:24:57.641618 24848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:24:57.642213 24848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:24:57.643395 24848 hybrid_clock.cc:648] HybridClock initialized: now 1786177497643371 us; error 35 us; skew 500 ppm
Aug 08 08:24:57 dist-test-slave-f7th krb5kdc[24259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177497, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:24:57.646049 24848 init.cc:377] Logged in from keytab as kudu/127.22.235.195@KRBTEST.COM (short username kudu)
I20260808 08:24:57.646973 24848 webserver.cc:460] Webserver started at http://127.22.235.195:36471/ using document root <none> and password file <none>
I20260808 08:24:57.647513 24848 fs_manager.cc:359] Metadata directory not provided
I20260808 08:24:57.647573 24848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:24:57.647773 24848 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:24:57.649767 24848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-2/data/instance:
uuid: "cee328890f4c4f5db7f79ba84c454b6d"
format_stamp: "Formatted at 2026-08-08 08:24:57 on dist-test-slave-f7th"
server_key: "cd147c5a10fc9258eca85356faad5a01"
server_key_iv: "d479f88f02c9b658886fc32242b0b099"
server_key_version: "encryptionkey@0"
I20260808 08:24:57.650279 24848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance:
uuid: "cee328890f4c4f5db7f79ba84c454b6d"
format_stamp: "Formatted at 2026-08-08 08:24:57 on dist-test-slave-f7th"
server_key: "cd147c5a10fc9258eca85356faad5a01"
server_key_iv: "d479f88f02c9b658886fc32242b0b099"
server_key_version: "encryptionkey@0"
I20260808 08:24:57.653860 24848 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 08:24:57.656111 24868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:24:57.657295 24848 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:24:57.657407 24848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-2/wal
uuid: "cee328890f4c4f5db7f79ba84c454b6d"
format_stamp: "Formatted at 2026-08-08 08:24:57 on dist-test-slave-f7th"
server_key: "cd147c5a10fc9258eca85356faad5a01"
server_key_iv: "d479f88f02c9b658886fc32242b0b099"
server_key_version: "encryptionkey@0"
I20260808 08:24:57.657511 24848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:24:57.675319 24848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:24:57.677919 24848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:24:57.678082 24848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:24:57.678627 24848 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:24:57.679409 24848 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:24:57.679454 24848 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:24:57.679502 24848 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:24:57.679534 24848 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:24:57.688017 24848 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.195:33335
I20260808 08:24:57.688067 25056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.195:33335 every 8 connection(s)
I20260808 08:24:57.688905 24848 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
I20260808 08:24:57.691504 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 24848
I20260808 08:24:57.691628 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance
I20260808 08:24:57.691939 23471 external_mini_cluster.cc:1428] Setting key e73e56703ad6b872c682797cd087702b
Aug 08 08:24:57 dist-test-slave-f7th krb5kdc[24259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177497, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.195@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:24:57.701316 24392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:24:57.690549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:60387 (local address 127.22.235.254:36873)
0808 08:24:57.690671 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 08:24:57.690674 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:24:57.691480 (+ 806us) server_negotiation.cc:407] Connection header received
0808 08:24:57.692914 (+ 1434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:24:57.692918 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:24:57.692986 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:24:57.693061 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:24:57.694362 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:24:57.694896 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:24:57.695942 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:24:57.696160 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:24:57.696229 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:24:57.698457 (+ 2228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:24:57.698460 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:24:57.698462 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:24:57.698488 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:24:57.699754 (+ 1266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:24:57.700148 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:24:57.700150 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:24:57.700152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:24:57.700221 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:24:57.700456 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:24:57.700458 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:24:57.700459 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:24:57.700635 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:24:57.700701 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 08:24:57.701106 (+ 405us) server_negotiation.cc:299] Negotiation successful
0808 08:24:57.701196 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 08:24:57.702090 25059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:24:57.690861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:36873 (local address 127.22.235.195:60387)
0808 08:24:57.691304 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0808 08:24:57.691360 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 08:24:57.692710 (+ 1350us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:24:57.693251 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:24:57.693260 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:24:57.693628 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:24:57.694226 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:24:57.694250 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:24:57.694984 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:24:57.694986 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:24:57.695855 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:24:57.695864 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:24:57.696265 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:24:57.696268 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:24:57.696398 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:24:57.696917 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:24:57.696933 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:24:57.698311 (+ 1378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:24:57.699841 (+ 1530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:24:57.699845 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:24:57.699855 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:24:57.700079 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:24:57.700281 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:24:57.700283 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:24:57.700284 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:24:57.700396 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:24:57.700705 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:24:57.700710 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:24:57.700927 (+ 217us) client_negotiation.cc:769] Sending connection context
0808 08:24:57.701073 (+ 146us) client_negotiation.cc:240] Negotiation successful
0808 08:24:57.701296 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":105,"threads_started":1}
I20260808 08:24:57.703131 25057 heartbeater.cc:347] Connected to a master server at 127.22.235.254:36873
I20260808 08:24:57.703341 25057 heartbeater.cc:464] Registering TS with master...
I20260808 08:24:57.703774 25057 heartbeater.cc:511] Master 127.22.235.254:36873 requested a full tablet report, sending...
I20260808 08:24:57.704799 24328 ts_manager.cc:194] Registered new tserver with Master: cee328890f4c4f5db7f79ba84c454b6d (127.22.235.195:33335)
I20260808 08:24:57.705489 24328 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.195@KRBTEST.COM'} at 127.22.235.195:60387
I20260808 08:24:57.705781 23471 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 08:24:57.715795 24392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:24:57.709142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36434 (local address 127.22.235.254:36873)
0808 08:24:57.709259 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 08:24:57.709263 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:24:57.709414 (+ 151us) server_negotiation.cc:407] Connection header received
0808 08:24:57.709542 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:24:57.709545 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:24:57.709583 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:24:57.709640 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:24:57.710303 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:24:57.710793 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:24:57.711570 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:24:57.711783 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:24:57.711864 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:24:57.713099 (+ 1235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:24:57.713103 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:24:57.713105 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:24:57.713129 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:24:57.714286 (+ 1157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:24:57.714682 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:24:57.714686 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:24:57.714687 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:24:57.714762 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:24:57.714999 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:24:57.715002 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:24:57.715004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:24:57.715200 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:24:57.715317 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 08:24:57.715552 (+ 235us) server_negotiation.cc:299] Negotiation successful
0808 08:24:57.715654 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 08:24:57.720242 24328 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36434:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 08:24:57.723347 24328 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 08:24:57.737100 25072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:24:57.731223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:36805 (local address 127.0.0.1:40452)
0808 08:24:57.731707 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0808 08:24:57.731758 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 08:24:57.731888 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:24:57.733002 (+ 1114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:24:57.733008 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:24:57.733030 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:24:57.733155 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:24:57.733162 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:24:57.734278 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:24:57.734281 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:24:57.735648 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:24:57.735659 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:24:57.736559 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:24:57.736562 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:24:57.736694 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:24:57.736735 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 08:24:57.736838 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 08:24:57.736959 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":2,"thread_start_us":81,"threads_started":1}
I20260808 08:24:57.737110 25075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:24:57.731382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40452 (local address 127.22.235.193:36805)
0808 08:24:57.731791 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0808 08:24:57.731809 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 08:24:57.731834 (+ 25us) server_negotiation.cc:407] Connection header received
0808 08:24:57.732551 (+ 717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:24:57.732557 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:24:57.732788 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:24:57.732992 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:24:57.733266 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:24:57.734156 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:24:57.735770 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:24:57.736451 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:24:57.736564 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:24:57.736736 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 08:24:57.736872 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 08:24:57.736961 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":129,"threads_started":1}
I20260808 08:24:57.737387 25074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:24:57.731905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.195:33335 (local address 127.0.0.1:49150)
0808 08:24:57.732272 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0808 08:24:57.732284 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 08:24:57.732373 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:24:57.733152 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:24:57.733155 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:24:57.733170 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:24:57.733262 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:24:57.733268 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:24:57.734500 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:24:57.734503 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:24:57.735757 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:24:57.735766 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:24:57.736911 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:24:57.736914 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:24:57.737020 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:24:57.737046 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 08:24:57.737106 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 08:24:57.737178 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"mutex_wait_us":105,"thread_start_us":124,"threads_started":1}
I20260808 08:24:57.737789 25076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:24:57.731350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.194:44851 (local address 127.0.0.1:44844)
0808 08:24:57.732227 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0808 08:24:57.732240 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:24:57.732325 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:24:57.733474 (+ 1149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:24:57.733477 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:24:57.733495 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:24:57.733583 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:24:57.733589 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:24:57.734803 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:24:57.734805 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:24:57.736028 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:24:57.736039 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:24:57.737427 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:24:57.737431 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:24:57.737527 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:24:57.737551 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 08:24:57.737607 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 08:24:57.737671 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":821,"mutex_wait_us":24,"thread_start_us":86,"threads_started":1}
I20260808 08:24:57.737892 25077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:24:57.732075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49150 (local address 127.22.235.195:33335)
0808 08:24:57.732524 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0808 08:24:57.732536 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 08:24:57.732559 (+ 23us) server_negotiation.cc:407] Connection header received
0808 08:24:57.732657 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:24:57.732662 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:24:57.732857 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:24:57.733007 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:24:57.733364 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:24:57.734359 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:24:57.735863 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:24:57.736574 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:24:57.736717 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:24:57.736843 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 08:24:57.737668 (+ 825us) server_negotiation.cc:299] Negotiation successful
0808 08:24:57.737764 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":152,"threads_started":1}
I20260808 08:24:57.740069 24515 tablet_service.cc:1467] Processing CreateTablet for tablet 7fa1bbd017f5400dba72800655a87493 (DEFAULT_TABLE table=test-table [id=82f26e9d96d14fb5ae8c6ed323cf7914]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:24:57.740202 24945 tablet_service.cc:1467] Processing CreateTablet for tablet 7fa1bbd017f5400dba72800655a87493 (DEFAULT_TABLE table=test-table [id=82f26e9d96d14fb5ae8c6ed323cf7914]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:24:57.740862 25080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:24:57.732270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44844 (local address 127.22.235.194:44851)
0808 08:24:57.732816 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0808 08:24:57.732827 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 08:24:57.732852 (+ 25us) server_negotiation.cc:407] Connection header received
0808 08:24:57.733009 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:24:57.733013 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:24:57.733178 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:24:57.733316 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:24:57.733708 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:24:57.734691 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:24:57.736152 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:24:57.736964 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:24:57.737075 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:24:57.737967 (+ 892us) server_negotiation.cc:1035] Waiting for connection context
0808 08:24:57.738025 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 08:24:57.740726 (+ 2701us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":129,"threads_started":1}
I20260808 08:24:57.741103 24515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fa1bbd017f5400dba72800655a87493. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:24:57.741169 24945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fa1bbd017f5400dba72800655a87493. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:24:57.743566 24739 tablet_service.cc:1467] Processing CreateTablet for tablet 7fa1bbd017f5400dba72800655a87493 (DEFAULT_TABLE table=test-table [id=82f26e9d96d14fb5ae8c6ed323cf7914]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:24:57.744592 24739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fa1bbd017f5400dba72800655a87493. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:24:57.747484 25081 tablet_bootstrap.cc:492] T 7fa1bbd017f5400dba72800655a87493 P 0be8f012a1574e399bcbadbebdec07a5: Bootstrap starting.
I20260808 08:24:57.748871 25082 tablet_bootstrap.cc:492] T 7fa1bbd017f5400dba72800655a87493 P cee328890f4c4f5db7f79ba84c454b6d: Bootstrap starting.
I20260808 08:24:57.750512 25081 tablet_bootstrap.cc:654] T 7fa1bbd017f5400dba72800655a87493 P 0be8f012a1574e399bcbadbebdec07a5: Neither blocks nor log segments found. Creating new log.
I20260808 08:24:57.750941 25082 tablet_bootstrap.cc:654] T 7fa1bbd017f5400dba72800655a87493 P cee328890f4c4f5db7f79ba84c454b6d: Neither blocks nor log segments found. Creating new log.
I20260808 08:24:57.752089 25082 log.cc:824] T 7fa1bbd017f5400dba72800655a87493 P cee328890f4c4f5db7f79ba84c454b6d: Log is configured to *not* fsync() on all Append() calls
I20260808 08:24:57.752089 25081 log.cc:824] T 7fa1bbd017f5400dba72800655a87493 P 0be8f012a1574e399bcbadbebdec07a5: Log is configured to *not* fsync() on all Append() calls
I20260808 08:24:57.752772 25083 tablet_bootstrap.cc:492] T 7fa1bbd017f5400dba72800655a87493 P 10a548dd59364a70a3246b6e14f7914b: Bootstrap starting.
I20260808 08:24:57.754034 25081 tablet_bootstrap.cc:492] T 7fa1bbd017f5400dba72800655a87493 P 0be8f012a1574e399bcbadbebdec07a5: No bootstrap required, opened a new log
I20260808 08:24:57.754170 25082 tablet_bootstrap.cc:492] T 7fa1bbd017f5400dba72800655a87493 P cee328890f4c4f5db7f79ba84c454b6d: No bootstrap required, opened a new log
I20260808 08:24:57.754230 25081 ts_tablet_manager.cc:1397] T 7fa1bbd017f5400dba72800655a87493 P 0be8f012a1574e399bcbadbebdec07a5: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260808 08:24:57.754356 25082 ts_tablet_manager.cc:1397] T 7fa1bbd017f5400dba72800655a87493 P cee328890f4c4f5db7f79ba84c454b6d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 08:24:57.755172 25083 tablet_bootstrap.cc:654] T 7fa1bbd017f5400dba72800655a87493 P 10a548dd59364a70a3246b6e14f7914b: Neither blocks nor log segments found. Creating new log.
I20260808 08:24:57.756006 25083 log.cc:824] T 7fa1bbd017f5400dba72800655a87493 P 10a548dd59364a70a3246b6e14f7914b: Log is configured to *not* fsync() on all Append() calls
I20260808 08:24:57.757445 25081 raft_consensus.cc:348] T 7fa1bbd017f5400dba72800655a87493 P 0be8f012a1574e399bcbadbebdec07a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be8f012a1574e399bcbadbebdec07a5" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 36805 } } peers { permanent_uuid: "10a548dd59364a70a3246b6e14f7914b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44851 } } peers { permanent_uuid: "cee328890f4c4f5db7f79ba84c454b6d" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 33335 } }
I20260808 08:24:57.757656 25081 raft_consensus.cc:374] T 7fa1bbd017f5400dba72800655a87493 P 0be8f012a1574e399bcbadbebdec07a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:24:57.757709 25081 raft_consensus.cc:729] T 7fa1bbd017f5400dba72800655a87493 P 0be8f012a1574e399bcbadbebdec07a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0be8f012a1574e399bcbadbebdec07a5, State: Initialized, Role: FOLLOWER
I20260808 08:24:57.758060 25083 tablet_bootstrap.cc:492] T 7fa1bbd017f5400dba72800655a87493 P 10a548dd59364a70a3246b6e14f7914b: No bootstrap required, opened a new log
I20260808 08:24:57.758106 25081 consensus_queue.cc:260] T 7fa1bbd017f5400dba72800655a87493 P 0be8f012a1574e399bcbadbebdec07a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be8f012a1574e399bcbadbebdec07a5" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 36805 } } peers { permanent_uuid: "10a548dd59364a70a3246b6e14f7914b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44851 } } peers { permanent_uuid: "cee328890f4c4f5db7f79ba84c454b6d" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 33335 } }
I20260808 08:24:57.758242 25083 ts_tablet_manager.cc:1397] T 7fa1bbd017f5400dba72800655a87493 P 10a548dd59364a70a3246b6e14f7914b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 08:24:57.758222 25082 raft_consensus.cc:348] T 7fa1bbd017f5400dba72800655a87493 P cee328890f4c4f5db7f79ba84c454b6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be8f012a1574e399bcbadbebdec07a5" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 36805 } } peers { permanent_uuid: "10a548dd59364a70a3246b6e14f7914b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44851 } } peers { permanent_uuid: "cee328890f4c4f5db7f79ba84c454b6d" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 33335 } }
I20260808 08:24:57.758488 25082 raft_consensus.cc:374] T 7fa1bbd017f5400dba72800655a87493 P cee328890f4c4f5db7f79ba84c454b6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:24:57.758558 25082 raft_consensus.cc:729] T 7fa1bbd017f5400dba72800655a87493 P cee328890f4c4f5db7f79ba84c454b6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cee328890f4c4f5db7f79ba84c454b6d, State: Initialized, Role: FOLLOWER
I20260808 08:24:57.758879 24624 heartbeater.cc:503] Master 127.22.235.254:36873 was elected leader, sending a full tablet report...
I20260808 08:24:57.759071 25081 ts_tablet_manager.cc:1428] T 7fa1bbd017f5400dba72800655a87493 P 0be8f012a1574e399bcbadbebdec07a5: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 08:24:57.759037 25082 consensus_queue.cc:260] T 7fa1bbd017f5400dba72800655a87493 P cee328890f4c4f5db7f79ba84c454b6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be8f012a1574e399bcbadbebdec07a5" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 36805 } } peers { permanent_uuid: "10a548dd59364a70a3246b6e14f7914b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44851 } } peers { permanent_uuid: "cee328890f4c4f5db7f79ba84c454b6d" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 33335 } }
I20260808 08:24:57.759884 25057 heartbeater.cc:503] Master 127.22.235.254:36873 was elected leader, sending a full tablet report...
I20260808 08:24:57.759949 25082 ts_tablet_manager.cc:1428] T 7fa1bbd017f5400dba72800655a87493 P cee328890f4c4f5db7f79ba84c454b6d: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 08:24:57.761678 25083 raft_consensus.cc:348] T 7fa1bbd017f5400dba72800655a87493 P 10a548dd59364a70a3246b6e14f7914b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be8f012a1574e399bcbadbebdec07a5" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 36805 } } peers { permanent_uuid: "10a548dd59364a70a3246b6e14f7914b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44851 } } peers { permanent_uuid: "cee328890f4c4f5db7f79ba84c454b6d" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 33335 } }
I20260808 08:24:57.761927 25083 raft_consensus.cc:374] T 7fa1bbd017f5400dba72800655a87493 P 10a548dd59364a70a3246b6e14f7914b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:24:57.761994 25083 raft_consensus.cc:729] T 7fa1bbd017f5400dba72800655a87493 P 10a548dd59364a70a3246b6e14f7914b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10a548dd59364a70a3246b6e14f7914b, State: Initialized, Role: FOLLOWER
I20260808 08:24:57.762427 25083 consensus_queue.cc:260] T 7fa1bbd017f5400dba72800655a87493 P 10a548dd59364a70a3246b6e14f7914b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be8f012a1574e399bcbadbebdec07a5" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 36805 } } peers { permanent_uuid: "10a548dd59364a70a3246b6e14f7914b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44851 } } peers { permanent_uuid: "cee328890f4c4f5db7f79ba84c454b6d" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 33335 } }
I20260808 08:24:57.763041 24841 heartbeater.cc:503] Master 127.22.235.254:36873 was elected leader, sending a full tablet report...
I20260808 08:24:57.763347 25083 ts_tablet_manager.cc:1428] T 7fa1bbd017f5400dba72800655a87493 P 10a548dd59364a70a3246b6e14f7914b: Time spent starting tablet: real 0.005s user 0.001s sys 0.005s
I20260808 08:24:57.828696 25090 raft_consensus.cc:482] T 7fa1bbd017f5400dba72800655a87493 P 10a548dd59364a70a3246b6e14f7914b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 08:24:57.828900 25090 raft_consensus.cc:504] T 7fa1bbd017f5400dba72800655a87493 P 10a548dd59364a70a3246b6e14f7914b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be8f012a1574e399bcbadbebdec07a5" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 36805 } } peers { permanent_uuid: "10a548dd59364a70a3246b6e14f7914b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44851 } } peers { permanent_uuid: "cee328890f4c4f5db7f79ba84c454b6d" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 33335 } }
I20260808 08:24:57.830063 25090 leader_election.cc:288] T 7fa1bbd017f5400dba72800655a87493 P 10a548dd59364a70a3246b6e14f7914b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0be8f012a1574e399bcbadbebdec07a5 (127.22.235.193:36805), cee328890f4c4f5db7f79ba84c454b6d (127.22.235.195:33335)
I20260808 08:24:57.834091 24843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:24:57.830312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.195:33335 (local address 127.22.235.194:57477)
0808 08:24:57.830439 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 08:24:57.830455 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 08:24:57.830565 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:24:57.830822 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:24:57.830825 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:24:57.830843 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:24:57.830954 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:24:57.830959 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:24:57.831843 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:24:57.831846 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:24:57.832999 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:24:57.833010 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:24:57.833735 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:24:57.833737 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:24:57.833818 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:24:57.833840 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 08:24:57.833897 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 08:24:57.833954 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 08:24:57.834131 25077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:24:57.830418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:57477 (local address 127.22.235.195:33335)
0808 08:24:57.830559 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 08:24:57.830562 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:24:57.830577 (+ 15us) server_negotiation.cc:407] Connection header received
0808 08:24:57.830678 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:24:57.830680 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:24:57.830722 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:24:57.830793 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:24:57.831064 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:24:57.831736 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:24:57.833114 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:24:57.833627 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:24:57.833741 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:24:57.833795 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 08:24:57.833931 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 08:24:57.834009 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 08:24:57.834450 25075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:24:57.830378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:60505 (local address 127.22.235.193:36805)
0808 08:24:57.830497 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 08:24:57.830501 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:24:57.831093 (+ 592us) server_negotiation.cc:407] Connection header received
0808 08:24:57.831169 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:24:57.831172 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:24:57.831223 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:24:57.831300 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:24:57.831550 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:24:57.832109 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:24:57.833380 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:24:57.833901 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:24:57.834031 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:24:57.834078 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 08:24:57.834279 (+ 201us) server_negotiation.cc:299] Negotiation successful
0808 08:24:57.834356 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 08:24:57.834798 25091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:24:57.830326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:36805 (local address 127.22.235.194:60505)
0808 08:24:57.831004 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0808 08:24:57.831015 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 08:24:57.831097 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:24:57.831319 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:24:57.831322 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:24:57.831337 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:24:57.831452 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:24:57.831457 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:24:57.832223 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:24:57.832227 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:24:57.833278 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:24:57.833286 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:24:57.834036 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:24:57.834039 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:24:57.834126 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:24:57.834149 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 08:24:57.834603 (+ 454us) client_negotiation.cc:240] Negotiation successful
0808 08:24:57.834675 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"mutex_wait_us":304,"thread_start_us":131,"threads_started":1}
I20260808 08:24:57.834792 24981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fa1bbd017f5400dba72800655a87493" candidate_uuid: "10a548dd59364a70a3246b6e14f7914b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cee328890f4c4f5db7f79ba84c454b6d" is_pre_election: true
I20260808 08:24:57.835144 24981 raft_consensus.cc:2454] T 7fa1bbd017f5400dba72800655a87493 P cee328890f4c4f5db7f79ba84c454b6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10a548dd59364a70a3246b6e14f7914b in term 0.
I20260808 08:24:57.835420 24552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fa1bbd017f5400dba72800655a87493" candidate_uuid: "10a548dd59364a70a3246b6e14f7914b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0be8f012a1574e399bcbadbebdec07a5" is_pre_election: true
I20260808 08:24:57.835642 24656 leader_election.cc:302] T 7fa1bbd017f5400dba72800655a87493 P 10a548dd59364a70a3246b6e14f7914b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10a548dd59364a70a3246b6e14f7914b, cee328890f4c4f5db7f79ba84c454b6d; no voters:
I20260808 08:24:57.835722 24552 raft_consensus.cc:2454] T 7fa1bbd017f5400dba72800655a87493 P 0be8f012a1574e399bcbadbebdec07a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10a548dd59364a70a3246b6e14f7914b in term 0.
I20260808 08:24:57.835899 25090 raft_consensus.cc:2781] T 7fa1bbd017f5400dba72800655a87493 P 10a548dd59364a70a3246b6e14f7914b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 08:24:57.835980 25090 raft_consensus.cc:482] T 7fa1bbd017f5400dba72800655a87493 P 10a548dd59364a70a3246b6e14f7914b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 08:24:57.836050 25090 raft_consensus.cc:3037] T 7fa1bbd017f5400dba72800655a87493 P 10a548dd59364a70a3246b6e14f7914b [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:24:57.837147 25090 raft_consensus.cc:504] T 7fa1bbd017f5400dba72800655a87493 P 10a548dd59364a70a3246b6e14f7914b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be8f012a1574e399bcbadbebdec07a5" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 36805 } } peers { permanent_uuid: "10a548dd59364a70a3246b6e14f7914b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44851 } } peers { permanent_uuid: "cee328890f4c4f5db7f79ba84c454b6d" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 33335 } }
I20260808 08:24:57.837476 25090 leader_election.cc:288] T 7fa1bbd017f5400dba72800655a87493 P 10a548dd59364a70a3246b6e14f7914b [CANDIDATE]: Term 1 election: Requested vote from peers 0be8f012a1574e399bcbadbebdec07a5 (127.22.235.193:36805), cee328890f4c4f5db7f79ba84c454b6d (127.22.235.195:33335)
I20260808 08:24:57.837751 24552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fa1bbd017f5400dba72800655a87493" candidate_uuid: "10a548dd59364a70a3246b6e14f7914b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0be8f012a1574e399bcbadbebdec07a5"
I20260808 08:24:57.837829 24981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7fa1bbd017f5400dba72800655a87493" candidate_uuid: "10a548dd59364a70a3246b6e14f7914b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cee328890f4c4f5db7f79ba84c454b6d"
I20260808 08:24:57.837859 24552 raft_consensus.cc:3037] T 7fa1bbd017f5400dba72800655a87493 P 0be8f012a1574e399bcbadbebdec07a5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:24:57.837951 24981 raft_consensus.cc:3037] T 7fa1bbd017f5400dba72800655a87493 P cee328890f4c4f5db7f79ba84c454b6d [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:24:57.838896 24552 raft_consensus.cc:2454] T 7fa1bbd017f5400dba72800655a87493 P 0be8f012a1574e399bcbadbebdec07a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10a548dd59364a70a3246b6e14f7914b in term 1.
I20260808 08:24:57.838927 24981 raft_consensus.cc:2454] T 7fa1bbd017f5400dba72800655a87493 P cee328890f4c4f5db7f79ba84c454b6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10a548dd59364a70a3246b6e14f7914b in term 1.
I20260808 08:24:57.839190 24652 leader_election.cc:302] T 7fa1bbd017f5400dba72800655a87493 P 10a548dd59364a70a3246b6e14f7914b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0be8f012a1574e399bcbadbebdec07a5, 10a548dd59364a70a3246b6e14f7914b; no voters:
I20260808 08:24:57.839346 25090 raft_consensus.cc:2781] T 7fa1bbd017f5400dba72800655a87493 P 10a548dd59364a70a3246b6e14f7914b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:24:57.839562 25090 raft_consensus.cc:686] T 7fa1bbd017f5400dba72800655a87493 P 10a548dd59364a70a3246b6e14f7914b [term 1 LEADER]: Becoming Leader. State: Replica: 10a548dd59364a70a3246b6e14f7914b, State: Running, Role: LEADER
I20260808 08:24:57.839887 25090 consensus_queue.cc:237] T 7fa1bbd017f5400dba72800655a87493 P 10a548dd59364a70a3246b6e14f7914b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be8f012a1574e399bcbadbebdec07a5" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 36805 } } peers { permanent_uuid: "10a548dd59364a70a3246b6e14f7914b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44851 } } peers { permanent_uuid: "cee328890f4c4f5db7f79ba84c454b6d" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 33335 } }
I20260808 08:24:57.843024 24327 catalog_manager.cc:5495] T 7fa1bbd017f5400dba72800655a87493 P 10a548dd59364a70a3246b6e14f7914b reported cstate change: term changed from 0 to 1, leader changed from <none> to 10a548dd59364a70a3246b6e14f7914b (127.22.235.194). New cstate: current_term: 1 leader_uuid: "10a548dd59364a70a3246b6e14f7914b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0be8f012a1574e399bcbadbebdec07a5" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 36805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10a548dd59364a70a3246b6e14f7914b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cee328890f4c4f5db7f79ba84c454b6d" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 33335 } health_report { overall_health: UNKNOWN } } }
Aug 08 08:24:57 dist-test-slave-f7th krb5kdc[24259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.235.193@KRBTEST.COM
I20260808 08:24:57.863161 25075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:24:57.853102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40454 (local address 127.22.235.193:36805)
0808 08:24:57.853270 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 08:24:57.853276 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 08:24:57.853410 (+ 134us) server_negotiation.cc:407] Connection header received
0808 08:24:57.853542 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:24:57.853545 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:24:57.853588 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:24:57.853654 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:24:57.854320 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:24:57.854817 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:24:57.855865 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:24:57.856217 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:24:57.856322 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:24:57.859162 (+ 2840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:24:57.859180 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:24:57.859201 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:24:57.859226 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:24:57.861282 (+ 2056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:24:57.861764 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:24:57.861772 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:24:57.861774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:24:57.861868 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:24:57.862130 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:24:57.862133 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:24:57.862135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:24:57.862407 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:24:57.862571 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 08:24:57.862894 (+ 323us) server_negotiation.cc:299] Negotiation successful
0808 08:24:57.863017 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Aug 08 08:24:57 dist-test-slave-f7th krb5kdc[24259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177497, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 08:24:57 dist-test-slave-f7th krb5kdc[24259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177497, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.235.193@KRBTEST.COM
I20260808 08:24:57.899384 25075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:24:57.890098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40456 (local address 127.22.235.193:36805)
0808 08:24:57.890221 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 08:24:57.890225 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:24:57.890409 (+ 184us) server_negotiation.cc:407] Connection header received
0808 08:24:57.890642 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:24:57.890647 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:24:57.890702 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:24:57.890767 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:24:57.891600 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:24:57.892140 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:24:57.893347 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:24:57.893570 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:24:57.893661 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:24:57.896169 (+ 2508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:24:57.896174 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:24:57.896177 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:24:57.896219 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:24:57.897610 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:24:57.898012 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:24:57.898016 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:24:57.898018 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:24:57.898102 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:24:57.898508 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:24:57.898511 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:24:57.898513 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:24:57.898696 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:24:57.898820 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 08:24:57.899103 (+ 283us) server_negotiation.cc:299] Negotiation successful
0808 08:24:57.899224 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
Aug 08 08:24:57 dist-test-slave-f7th krb5kdc[24259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177497, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 08:24:57 dist-test-slave-f7th krb5kdc[24259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177497, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.235.193@KRBTEST.COM
I20260808 08:24:57.927438 25075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:24:57.920046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40472 (local address 127.22.235.193:36805)
0808 08:24:57.920229 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 08:24:57.920233 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:24:57.920306 (+ 73us) server_negotiation.cc:407] Connection header received
0808 08:24:57.920426 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:24:57.920428 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:24:57.920486 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:24:57.920555 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:24:57.921222 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:24:57.921710 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:24:57.922512 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:24:57.922705 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:24:57.922770 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:24:57.924940 (+ 2170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:24:57.924943 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:24:57.924945 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:24:57.924969 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:24:57.926222 (+ 1253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:24:57.926540 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:24:57.926543 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:24:57.926544 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:24:57.926607 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:24:57.926782 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:24:57.926783 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:24:57.926784 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:24:57.926943 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:24:57.927044 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 08:24:57.927186 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 08:24:57.927276 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260808 08:24:57.928941 24462 tablet_service.cc:2908] Rejecting scan request for tablet 7fa1bbd017f5400dba72800655a87493: Uninitialized: clean time has not yet been initialized
I20260808 08:24:57.930997 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 24427
W20260808 08:24:57.940788 25058 tablet.cc:2369] T 7fa1bbd017f5400dba72800655a87493 P cee328890f4c4f5db7f79ba84c454b6d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 08:24:57.949383 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 24631
I20260808 08:24:57.960939 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 24848
I20260808 08:24:57.976594 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 24283
2026-08-08T08:24:57Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2459 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 08:24:58 dist-test-slave-f7th krb5kdc[25125](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 08:24:58 dist-test-slave-f7th krb5kdc[25125](info): set up 2 sockets
Aug 08 08:24:58 dist-test-slave-f7th krb5kdc[25125](info): commencing operation
krb5kdc: starting...
W20260808 08:24:59.142984 23471 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Aug 08 08:24:59 dist-test-slave-f7th krb5kdc[25125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177499, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T08:24:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:24:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.235.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
I20260808 08:24:59.261566 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:46093
--webserver_interface=127.22.235.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:39441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:46093
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:24:59.372231 25144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:24:59.372473 25144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:24:59.372517 25144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:24:59.376857 25144 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:24:59.376914 25144 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 08:24:59.376930 25144 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:24:59.376947 25144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:24:59.376961 25144 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:24:59.376977 25144 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:24:59.382400 25144 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:39441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:46093
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:46093
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.235.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25144
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:24:59.383390 25144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:24:59.384387 25144 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:24:59.392724 25150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:24:59.392777 25144 server_base.cc:1034] running on GCE node
W20260808 08:24:59.392694 25154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:24:59.392702 25151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:24:59.393501 25144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:24:59.394533 25144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:24:59.395692 25144 hybrid_clock.cc:648] HybridClock initialized: now 1786177499395679 us; error 24 us; skew 500 ppm
Aug 08 08:24:59 dist-test-slave-f7th krb5kdc[25125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177499, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:24:59.401341 25144 init.cc:377] Logged in from keytab as kudu/127.22.235.254@KRBTEST.COM (short username kudu)
I20260808 08:24:59.402194 25144 webserver.cc:460] Webserver started at http://127.22.235.254:35263/ using document root <none> and password file <none>
I20260808 08:24:59.402709 25144 fs_manager.cc:359] Metadata directory not provided
I20260808 08:24:59.402763 25144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:24:59.402932 25144 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:24:59.404902 25144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "f3651710b85e4fe9be5fd28aff89bf89"
format_stamp: "Formatted at 2026-08-08 08:24:59 on dist-test-slave-f7th"
server_key: "597d5247b87d65c4bb73bd38dc608fa2"
server_key_iv: "ca121449bc6d16409a6f73503e54b478"
server_key_version: "encryptionkey@0"
I20260808 08:24:59.405412 25144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "f3651710b85e4fe9be5fd28aff89bf89"
format_stamp: "Formatted at 2026-08-08 08:24:59 on dist-test-slave-f7th"
server_key: "597d5247b87d65c4bb73bd38dc608fa2"
server_key_iv: "ca121449bc6d16409a6f73503e54b478"
server_key_version: "encryptionkey@0"
I20260808 08:24:59.409051 25144 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260808 08:24:59.411433 25163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:24:59.412747 25144 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:24:59.412848 25144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "f3651710b85e4fe9be5fd28aff89bf89"
format_stamp: "Formatted at 2026-08-08 08:24:59 on dist-test-slave-f7th"
server_key: "597d5247b87d65c4bb73bd38dc608fa2"
server_key_iv: "ca121449bc6d16409a6f73503e54b478"
server_key_version: "encryptionkey@0"
I20260808 08:24:59.412935 25144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:24:59.423287 25144 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:24:59.425947 25144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:24:59.426095 25144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:24:59.433728 25144 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.254:46093
I20260808 08:24:59.433766 25265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.254:46093 every 8 connection(s)
I20260808 08:24:59.434653 25144 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:24:59.437204 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 25144
I20260808 08:24:59.437345 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:24:59.437619 25269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:24:59.437636 23471 external_mini_cluster.cc:1428] Setting key 7357786d92574fee91599712f64aa588
I20260808 08:24:59.443774 25269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3651710b85e4fe9be5fd28aff89bf89: Bootstrap starting.
I20260808 08:24:59.446097 25269 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3651710b85e4fe9be5fd28aff89bf89: Neither blocks nor log segments found. Creating new log.
Aug 08 08:24:59 dist-test-slave-f7th krb5kdc[25125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177499, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:24:59.446818 25269 log.cc:824] T 00000000000000000000000000000000 P f3651710b85e4fe9be5fd28aff89bf89: Log is configured to *not* fsync() on all Append() calls
I20260808 08:24:59.448587 25269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3651710b85e4fe9be5fd28aff89bf89: No bootstrap required, opened a new log
I20260808 08:24:59.451501 25273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:24:59.439058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33396 (local address 127.22.235.254:46093)
0808 08:24:59.439481 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0808 08:24:59.439496 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 08:24:59.439532 (+ 36us) server_negotiation.cc:407] Connection header received
0808 08:24:59.440286 (+ 754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:24:59.440305 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:24:59.440676 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:24:59.441023 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:24:59.441771 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:24:59.442761 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:24:59.443584 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:24:59.444078 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:24:59.444225 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:24:59.446660 (+ 2435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:24:59.446666 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:24:59.446680 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:24:59.446707 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:24:59.448875 (+ 2168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:24:59.449286 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:24:59.449291 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:24:59.449295 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:24:59.449421 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:24:59.449679 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:24:59.449682 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:24:59.449683 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:24:59.450054 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:24:59.450226 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 08:24:59.450521 (+ 295us) server_negotiation.cc:299] Negotiation successful
0808 08:24:59.450786 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":137,"threads_started":1}
I20260808 08:24:59.451524 25269 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3651710b85e4fe9be5fd28aff89bf89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3651710b85e4fe9be5fd28aff89bf89" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46093 } }
I20260808 08:24:59.451795 25269 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3651710b85e4fe9be5fd28aff89bf89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:24:59.451865 25269 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3651710b85e4fe9be5fd28aff89bf89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3651710b85e4fe9be5fd28aff89bf89, State: Initialized, Role: FOLLOWER
I20260808 08:24:59.452347 25269 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3651710b85e4fe9be5fd28aff89bf89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3651710b85e4fe9be5fd28aff89bf89" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46093 } }
I20260808 08:24:59.452504 25269 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3651710b85e4fe9be5fd28aff89bf89 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:24:59.452570 25269 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3651710b85e4fe9be5fd28aff89bf89 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:24:59.452680 25269 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3651710b85e4fe9be5fd28aff89bf89 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:24:59.453539 25269 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3651710b85e4fe9be5fd28aff89bf89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3651710b85e4fe9be5fd28aff89bf89" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46093 } }
I20260808 08:24:59.453881 25269 leader_election.cc:302] T 00000000000000000000000000000000 P f3651710b85e4fe9be5fd28aff89bf89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3651710b85e4fe9be5fd28aff89bf89; no voters:
I20260808 08:24:59.454170 25269 leader_election.cc:288] T 00000000000000000000000000000000 P f3651710b85e4fe9be5fd28aff89bf89 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:24:59.454272 25276 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3651710b85e4fe9be5fd28aff89bf89 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:24:59.454469 25276 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3651710b85e4fe9be5fd28aff89bf89 [term 1 LEADER]: Becoming Leader. State: Replica: f3651710b85e4fe9be5fd28aff89bf89, State: Running, Role: LEADER
I20260808 08:24:59.454763 25276 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3651710b85e4fe9be5fd28aff89bf89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3651710b85e4fe9be5fd28aff89bf89" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46093 } }
I20260808 08:24:59.455305 25269 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3651710b85e4fe9be5fd28aff89bf89 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:24:59.456298 25277 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3651710b85e4fe9be5fd28aff89bf89 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f3651710b85e4fe9be5fd28aff89bf89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3651710b85e4fe9be5fd28aff89bf89" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46093 } } }
I20260808 08:24:59.456422 25277 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3651710b85e4fe9be5fd28aff89bf89 [sys.catalog]: This master's current role is: LEADER
I20260808 08:24:59.456732 25278 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3651710b85e4fe9be5fd28aff89bf89 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3651710b85e4fe9be5fd28aff89bf89. Latest consensus state: current_term: 1 leader_uuid: "f3651710b85e4fe9be5fd28aff89bf89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3651710b85e4fe9be5fd28aff89bf89" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46093 } } }
I20260808 08:24:59.456823 25278 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3651710b85e4fe9be5fd28aff89bf89 [sys.catalog]: This master's current role is: LEADER
I20260808 08:24:59.456928 25290 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:24:59.459962 25290 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:24:59.465416 25290 catalog_manager.cc:1347] Generated new cluster ID: 8352df7fd5c74cbeaa11d002e65aed5c
I20260808 08:24:59.465469 25290 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:24:59.489681 25290 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:24:59.490644 25290 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:24:59.498814 25290 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3651710b85e4fe9be5fd28aff89bf89: Generated new TSK 0
I20260808 08:24:59.499491 25290 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.235.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab.
Entry for principal kudu/127.22.235.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab.
I20260808 08:24:59.525856 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.193:0
--local_ip_for_outbound_sockets=127.22.235.193
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:46093
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab
--principal=kudu/127.22.235.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:39441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:24:59.636979 25307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:24:59.637236 25307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:24:59.637305 25307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:24:59.641314 25307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:24:59.641373 25307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:24:59.641426 25307 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 08:24:59.641476 25307 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.193
I20260808 08:24:59.646901 25307 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:39441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab
--principal=kudu/127.22.235.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.235.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.235.254:46093
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25307
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.193
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:24:59.647940 25307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:24:59.649006 25307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:24:59.658696 25313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:24:59.658802 25307 server_base.cc:1034] running on GCE node
W20260808 08:24:59.658672 25315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:24:59.658717 25312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:24:59.659261 25307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:24:59.659847 25307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:24:59.660996 25307 hybrid_clock.cc:648] HybridClock initialized: now 1786177499660973 us; error 32 us; skew 500 ppm
Aug 08 08:24:59 dist-test-slave-f7th krb5kdc[25125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177499, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:24:59.663573 25307 init.cc:377] Logged in from keytab as kudu/127.22.235.193@KRBTEST.COM (short username kudu)
I20260808 08:24:59.664369 25307 webserver.cc:460] Webserver started at http://127.22.235.193:36783/ using document root <none> and password file <none>
I20260808 08:24:59.664937 25307 fs_manager.cc:359] Metadata directory not provided
I20260808 08:24:59.665024 25307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:24:59.665225 25307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:24:59.667194 25307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-0/data/instance:
uuid: "8a07e1d1baaf41689220531bd2692f11"
format_stamp: "Formatted at 2026-08-08 08:24:59 on dist-test-slave-f7th"
server_key: "5871d31129a70da8197406be0faa53ad"
server_key_iv: "57e890461a29e1966e60e684aed49528"
server_key_version: "encryptionkey@0"
I20260808 08:24:59.667706 25307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance:
uuid: "8a07e1d1baaf41689220531bd2692f11"
format_stamp: "Formatted at 2026-08-08 08:24:59 on dist-test-slave-f7th"
server_key: "5871d31129a70da8197406be0faa53ad"
server_key_iv: "57e890461a29e1966e60e684aed49528"
server_key_version: "encryptionkey@0"
I20260808 08:24:59.671217 25307 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260808 08:24:59.673554 25329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:24:59.674768 25307 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 08:24:59.674880 25307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-0/wal
uuid: "8a07e1d1baaf41689220531bd2692f11"
format_stamp: "Formatted at 2026-08-08 08:24:59 on dist-test-slave-f7th"
server_key: "5871d31129a70da8197406be0faa53ad"
server_key_iv: "57e890461a29e1966e60e684aed49528"
server_key_version: "encryptionkey@0"
I20260808 08:24:59.674984 25307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:24:59.717200 25307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:24:59.720043 25307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:24:59.720224 25307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:24:59.721760 25307 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:24:59.721814 25307 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:24:59.721871 25307 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:24:59.721899 25307 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 08:24:59 dist-test-slave-f7th krb5kdc[25125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177499, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:24:59.732038 25307 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.193:42107
I20260808 08:24:59.732106 25516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.193:42107 every 8 connection(s)
I20260808 08:24:59.733125 25307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
I20260808 08:24:59.734908 25273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:24:59.722589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:51687 (local address 127.22.235.254:46093)
0808 08:24:59.722708 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 08:24:59.722711 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:24:59.723781 (+ 1070us) server_negotiation.cc:407] Connection header received
0808 08:24:59.724843 (+ 1062us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:24:59.724848 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:24:59.724912 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:24:59.724972 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:24:59.726613 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:24:59.727432 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:24:59.728681 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:24:59.728918 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:24:59.729013 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:24:59.731344 (+ 2331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:24:59.731348 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:24:59.731352 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:24:59.731383 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:24:59.733024 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:24:59.733511 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:24:59.733515 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:24:59.733517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:24:59.733623 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:24:59.733997 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:24:59.734002 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:24:59.734003 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:24:59.734197 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:24:59.734276 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 08:24:59.734690 (+ 414us) server_negotiation.cc:299] Negotiation successful
0808 08:24:59.734787 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 08:24:59.735632 25344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:24:59.723129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46093 (local address 127.22.235.193:51687)
0808 08:24:59.723634 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0808 08:24:59.723677 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 08:24:59.724567 (+ 890us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:24:59.725391 (+ 824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:24:59.725401 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:24:59.725765 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:24:59.726419 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:24:59.726447 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:24:59.727593 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:24:59.727597 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:24:59.728535 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:24:59.728546 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:24:59.729073 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:24:59.729078 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:24:59.729240 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:24:59.729739 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:24:59.729755 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:24:59.731174 (+ 1419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:24:59.733131 (+ 1957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:24:59.733136 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:24:59.733146 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:24:59.733410 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:24:59.733733 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:24:59.733736 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:24:59.733738 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:24:59.733901 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:24:59.734290 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:24:59.734295 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:24:59.734507 (+ 212us) client_negotiation.cc:769] Sending connection context
0808 08:24:59.734655 (+ 148us) client_negotiation.cc:240] Negotiation successful
0808 08:24:59.734884 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":156,"threads_started":1}
I20260808 08:24:59.736835 25518 heartbeater.cc:347] Connected to a master server at 127.22.235.254:46093
I20260808 08:24:59.736985 25518 heartbeater.cc:464] Registering TS with master...
I20260808 08:24:59.737434 25518 heartbeater.cc:511] Master 127.22.235.254:46093 requested a full tablet report, sending...
I20260808 08:24:59.739004 25188 ts_manager.cc:194] Registered new tserver with Master: 8a07e1d1baaf41689220531bd2692f11 (127.22.235.193:42107)
I20260808 08:24:59.740286 25188 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.193@KRBTEST.COM'} at 127.22.235.193:51687
I20260808 08:24:59.742614 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 25307
I20260808 08:24:59.742715 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance
I20260808 08:24:59.742965 23471 external_mini_cluster.cc:1428] Setting key 725bf93b038d2782335e2c9425807987
I20260808 08:24:59.745404 25273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:24:59.738849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:34493 (local address 127.22.235.254:46093)
0808 08:24:59.738986 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 08:24:59.738990 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:24:59.739128 (+ 138us) server_negotiation.cc:407] Connection header received
0808 08:24:59.739246 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:24:59.739249 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:24:59.739287 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:24:59.739342 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:24:59.739794 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:24:59.740282 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:24:59.741254 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:24:59.741485 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:24:59.741577 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:24:59.742559 (+ 982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:24:59.742563 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:24:59.742565 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:24:59.742589 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:24:59.743879 (+ 1290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:24:59.744305 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:24:59.744308 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:24:59.744310 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:24:59.744401 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:24:59.744663 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:24:59.744666 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:24:59.744668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:24:59.744834 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:24:59.744919 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 08:24:59.745162 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 08:24:59.745261 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 08:24:59.745414 25531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:24:59.738780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46093 (local address 127.22.235.193:34493)
0808 08:24:59.739048 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0808 08:24:59.739064 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 08:24:59.739159 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:24:59.739377 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:24:59.739379 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:24:59.739568 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:24:59.739682 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:24:59.739690 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:24:59.740401 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:24:59.740405 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:24:59.741131 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:24:59.741141 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:24:59.741590 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:24:59.741594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:24:59.741698 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:24:59.742109 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:24:59.742120 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:24:59.742428 (+ 308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:24:59.743975 (+ 1547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:24:59.743978 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:24:59.743981 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:24:59.744223 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:24:59.744481 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:24:59.744484 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:24:59.744486 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:24:59.744565 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:24:59.744920 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:24:59.744923 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:24:59.745017 (+ 94us) client_negotiation.cc:769] Sending connection context
0808 08:24:59.745101 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 08:24:59.745228 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":2,"thread_start_us":102,"threads_started":1}
WARNING: no policy specified for kudu/127.22.235.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab.
Entry for principal kudu/127.22.235.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab.
I20260808 08:24:59.768934 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.194:0
--local_ip_for_outbound_sockets=127.22.235.194
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:46093
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab
--principal=kudu/127.22.235.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:39441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:24:59.882827 25536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:24:59.883073 25536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:24:59.883113 25536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:24:59.887094 25536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:24:59.887147 25536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:24:59.887182 25536 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 08:24:59.887218 25536 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.194
I20260808 08:24:59.892228 25536 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:39441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab
--principal=kudu/127.22.235.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.235.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.235.254:46093
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25536
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.194
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:24:59.893190 25536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:24:59.894146 25536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:24:59.903643 25549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:24:59.903637 25551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:24:59.903690 25536 server_base.cc:1034] running on GCE node
W20260808 08:24:59.903637 25547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:24:59.904274 25536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:24:59.904963 25536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:24:59.906138 25536 hybrid_clock.cc:648] HybridClock initialized: now 1786177499906110 us; error 27 us; skew 500 ppm
Aug 08 08:24:59 dist-test-slave-f7th krb5kdc[25125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177499, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:24:59.908874 25536 init.cc:377] Logged in from keytab as kudu/127.22.235.194@KRBTEST.COM (short username kudu)
I20260808 08:24:59.909720 25536 webserver.cc:460] Webserver started at http://127.22.235.194:35731/ using document root <none> and password file <none>
I20260808 08:24:59.910281 25536 fs_manager.cc:359] Metadata directory not provided
I20260808 08:24:59.910351 25536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:24:59.910552 25536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:24:59.912670 25536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-1/data/instance:
uuid: "15380f70591f440a810a5e6e46fa89ad"
format_stamp: "Formatted at 2026-08-08 08:24:59 on dist-test-slave-f7th"
server_key: "dacf6a72429fe2a8dc5c91ee736e03fa"
server_key_iv: "6a4260d75841aca4f8066cad38421adb"
server_key_version: "encryptionkey@0"
I20260808 08:24:59.913220 25536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance:
uuid: "15380f70591f440a810a5e6e46fa89ad"
format_stamp: "Formatted at 2026-08-08 08:24:59 on dist-test-slave-f7th"
server_key: "dacf6a72429fe2a8dc5c91ee736e03fa"
server_key_iv: "6a4260d75841aca4f8066cad38421adb"
server_key_version: "encryptionkey@0"
I20260808 08:24:59.917023 25536 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 08:24:59.919500 25566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:24:59.920859 25536 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:24:59.920974 25536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-1/wal
uuid: "15380f70591f440a810a5e6e46fa89ad"
format_stamp: "Formatted at 2026-08-08 08:24:59 on dist-test-slave-f7th"
server_key: "dacf6a72429fe2a8dc5c91ee736e03fa"
server_key_iv: "6a4260d75841aca4f8066cad38421adb"
server_key_version: "encryptionkey@0"
I20260808 08:24:59.921062 25536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:24:59.938310 25536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:24:59.941123 25536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:24:59.941303 25536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:24:59.942792 25536 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:24:59.942852 25536 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:24:59.942906 25536 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:24:59.942939 25536 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 08:24:59 dist-test-slave-f7th krb5kdc[25125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177499, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.194@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:24:59.953460 25536 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.194:44277
I20260808 08:24:59.953752 25752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.194:44277 every 8 connection(s)
I20260808 08:24:59.954470 25536 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
I20260808 08:24:59.955792 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 25536
I20260808 08:24:59.955900 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance
I20260808 08:24:59.956224 23471 external_mini_cluster.cc:1428] Setting key f0e5405868b5c882f676bbc4594429d0
I20260808 08:24:59.956735 25273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:24:59.943478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:50515 (local address 127.22.235.254:46093)
0808 08:24:59.943600 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 08:24:59.943606 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 08:24:59.945267 (+ 1661us) server_negotiation.cc:407] Connection header received
0808 08:24:59.946507 (+ 1240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:24:59.946511 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:24:59.946575 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:24:59.946708 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:24:59.948204 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:24:59.948926 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:24:59.950139 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:24:59.950359 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:24:59.950449 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:24:59.953087 (+ 2638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:24:59.953091 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:24:59.953093 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:24:59.953123 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:24:59.954545 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:24:59.955097 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:24:59.955102 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:24:59.955104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:24:59.955207 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:24:59.955556 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:24:59.955558 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:24:59.955560 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:24:59.955757 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:24:59.955845 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 08:24:59.956450 (+ 605us) server_negotiation.cc:299] Negotiation successful
0808 08:24:59.956558 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 08:24:59.957649 25579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:24:59.944542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46093 (local address 127.22.235.194:50515)
0808 08:24:59.945116 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0808 08:24:59.945158 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 08:24:59.946249 (+ 1091us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:24:59.946923 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:24:59.946932 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:24:59.947393 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:24:59.948058 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:24:59.948080 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:24:59.949058 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:24:59.949061 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:24:59.950016 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:24:59.950027 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:24:59.950534 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:24:59.950539 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:24:59.950721 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:24:59.951239 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:24:59.951255 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:24:59.952901 (+ 1646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:24:59.954644 (+ 1743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:24:59.954650 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:24:59.954664 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:24:59.954936 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:24:59.955306 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:24:59.955310 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:24:59.955312 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:24:59.955463 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:24:59.955868 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:24:59.955874 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:24:59.956168 (+ 294us) client_negotiation.cc:769] Sending connection context
0808 08:24:59.956368 (+ 200us) client_negotiation.cc:240] Negotiation successful
0808 08:24:59.956715 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":121,"threads_started":1}
I20260808 08:24:59.958936 25754 heartbeater.cc:347] Connected to a master server at 127.22.235.254:46093
I20260808 08:24:59.959154 25754 heartbeater.cc:464] Registering TS with master...
I20260808 08:24:59.959745 25754 heartbeater.cc:511] Master 127.22.235.254:46093 requested a full tablet report, sending...
I20260808 08:24:59.960863 25190 ts_manager.cc:194] Registered new tserver with Master: 15380f70591f440a810a5e6e46fa89ad (127.22.235.194:44277)
I20260808 08:24:59.961597 25190 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.194@KRBTEST.COM'} at 127.22.235.194:50515
WARNING: no policy specified for kudu/127.22.235.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.195@KRBTEST.COM" created.
I20260808 08:24:59.973171 25774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:24:59.960938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46093 (local address 127.22.235.194:46351)
0808 08:24:59.967051 (+ 6113us) negotiation.cc:107] Waiting for socket to connect
0808 08:24:59.967063 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 08:24:59.967163 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:24:59.967377 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:24:59.967380 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:24:59.967557 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:24:59.967647 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:24:59.967652 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:24:59.968355 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:24:59.968359 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:24:59.968967 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:24:59.968976 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:24:59.969421 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:24:59.969425 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:24:59.969508 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:24:59.969957 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:24:59.969968 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:24:59.970255 (+ 287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:24:59.971734 (+ 1479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:24:59.971737 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:24:59.971739 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:24:59.971929 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:24:59.972169 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:24:59.972172 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:24:59.972174 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:24:59.972257 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:24:59.972663 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:24:59.972666 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:24:59.972759 (+ 93us) client_negotiation.cc:769] Sending connection context
0808 08:24:59.972854 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 08:24:59.972984 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6030,"thread_start_us":89,"threads_started":1}
I20260808 08:24:59.973165 25273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:24:59.961021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:46351 (local address 127.22.235.254:46093)
0808 08:24:59.961139 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 08:24:59.961143 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:24:59.967136 (+ 5993us) server_negotiation.cc:407] Connection header received
0808 08:24:59.967254 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:24:59.967256 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:24:59.967294 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:24:59.967360 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:24:59.967750 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:24:59.968235 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:24:59.969077 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:24:59.969304 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:24:59.969393 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:24:59.970388 (+ 995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:24:59.970392 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:24:59.970394 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:24:59.970419 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:24:59.971637 (+ 1218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:24:59.972015 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:24:59.972018 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:24:59.972020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:24:59.972091 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:24:59.972343 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:24:59.972346 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:24:59.972347 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:24:59.972547 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:24:59.972657 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 08:24:59.972909 (+ 252us) server_negotiation.cc:299] Negotiation successful
0808 08:24:59.973014 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab.
Entry for principal kudu/127.22.235.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab.
I20260808 08:24:59.982842 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.195:0
--local_ip_for_outbound_sockets=127.22.235.195
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:46093
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab
--principal=kudu/127.22.235.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:39441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:25:00.096125 25783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:00.096369 25783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:00.096412 25783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:00.100541 25783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:00.100595 25783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:00.100659 25783 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 08:25:00.100714 25783 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.195
I20260808 08:25:00.105813 25783 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:39441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab
--principal=kudu/127.22.235.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.235.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.235.254:46093
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25783
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.195
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:00.106765 25783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:00.107748 25783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:00.117125 25789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:00.117162 25788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:00.117179 25791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:00.117419 25783 server_base.cc:1034] running on GCE node
I20260808 08:25:00.117794 25783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:00.118430 25783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:00.119586 25783 hybrid_clock.cc:648] HybridClock initialized: now 1786177500119558 us; error 39 us; skew 500 ppm
Aug 08 08:25:00 dist-test-slave-f7th krb5kdc[25125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177500, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:00.122236 25783 init.cc:377] Logged in from keytab as kudu/127.22.235.195@KRBTEST.COM (short username kudu)
I20260808 08:25:00.123118 25783 webserver.cc:460] Webserver started at http://127.22.235.195:39939/ using document root <none> and password file <none>
I20260808 08:25:00.123658 25783 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:00.123737 25783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:00.123945 25783 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:00.125988 25783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-2/data/instance:
uuid: "60a28860872c4a078b71afee9b53ace1"
format_stamp: "Formatted at 2026-08-08 08:25:00 on dist-test-slave-f7th"
server_key: "4ce29a422bbb417ceb68e8b7c7eeb012"
server_key_iv: "b94003d4e95da091bb019cd070d25f7e"
server_key_version: "encryptionkey@0"
I20260808 08:25:00.126515 25783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance:
uuid: "60a28860872c4a078b71afee9b53ace1"
format_stamp: "Formatted at 2026-08-08 08:25:00 on dist-test-slave-f7th"
server_key: "4ce29a422bbb417ceb68e8b7c7eeb012"
server_key_iv: "b94003d4e95da091bb019cd070d25f7e"
server_key_version: "encryptionkey@0"
I20260808 08:25:00.130102 25783 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 08:25:00.132354 25799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:00.133566 25783 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:25:00.133682 25783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-2/wal
uuid: "60a28860872c4a078b71afee9b53ace1"
format_stamp: "Formatted at 2026-08-08 08:25:00 on dist-test-slave-f7th"
server_key: "4ce29a422bbb417ceb68e8b7c7eeb012"
server_key_iv: "b94003d4e95da091bb019cd070d25f7e"
server_key_version: "encryptionkey@0"
I20260808 08:25:00.133775 25783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:00.146083 25783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:00.148903 25783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:00.149081 25783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:00.150532 25783 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:25:00.150581 25783 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:00.150622 25783 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:25:00.150650 25783 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 08:25:00 dist-test-slave-f7th krb5kdc[25125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177500, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.195@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:25:00.160828 25783 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.195:39991
I20260808 08:25:00.160944 25994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.195:39991 every 8 connection(s)
I20260808 08:25:00.161686 25783 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
I20260808 08:25:00.163962 25273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:00.151168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:35637 (local address 127.22.235.254:46093)
0808 08:25:00.151323 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:00.151328 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:00.152642 (+ 1314us) server_negotiation.cc:407] Connection header received
0808 08:25:00.153916 (+ 1274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:00.153920 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:00.153968 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:00.154046 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:00.155676 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.156379 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.157680 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.157906 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.157994 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:00.160684 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:00.160689 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:00.160692 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:00.160723 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:00.162242 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:00.162768 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:00.162771 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:00.162772 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:00.162853 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:00.163106 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:00.163108 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:00.163109 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:00.163274 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:00.163349 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:00.163742 (+ 393us) server_negotiation.cc:299] Negotiation successful
0808 08:25:00.163843 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 08:25:00.164800 25811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:00.151962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46093 (local address 127.22.235.195:35637)
0808 08:25:00.152446 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:00.152515 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:00.153685 (+ 1170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:00.154252 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:00.154262 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:00.154754 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:00.155443 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:00.155477 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.156504 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.156507 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:00.157565 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:00.157576 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.158050 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.158054 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:00.158215 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:00.158744 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:00.158765 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:00.160435 (+ 1670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:00.162370 (+ 1935us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:00.162376 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:00.162389 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:00.162676 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:00.162919 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:00.162921 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:00.162922 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:00.163042 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:00.163355 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:00.163360 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:00.163564 (+ 204us) client_negotiation.cc:769] Sending connection context
0808 08:25:00.163705 (+ 141us) client_negotiation.cc:240] Negotiation successful
0808 08:25:00.163928 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":99,"threads_started":1}
I20260808 08:25:00.165922 25995 heartbeater.cc:347] Connected to a master server at 127.22.235.254:46093
I20260808 08:25:00.166061 25995 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:00.166518 25995 heartbeater.cc:511] Master 127.22.235.254:46093 requested a full tablet report, sending...
I20260808 08:25:00.167521 25188 ts_manager.cc:194] Registered new tserver with Master: 60a28860872c4a078b71afee9b53ace1 (127.22.235.195:39991)
I20260808 08:25:00.168208 25188 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.195@KRBTEST.COM'} at 127.22.235.195:35637
I20260808 08:25:00.169251 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 25783
I20260808 08:25:00.169369 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance
I20260808 08:25:00.169636 23471 external_mini_cluster.cc:1428] Setting key 66c8b06801916b56c142c29dedc49a38
I20260808 08:25:00.171602 23471 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 08:25:00.174975 26002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:00.167891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46093 (local address 127.22.235.195:45973)
0808 08:25:00.168150 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:00.168163 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:00.168257 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:00.168543 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:00.168545 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:00.168766 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:00.168867 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:00.168873 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.169829 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.169832 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:00.170594 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:00.170605 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.171045 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.171048 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:00.171128 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:00.171580 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:00.171593 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:00.171953 (+ 360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:00.173584 (+ 1631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:00.173588 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:00.173590 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:00.173822 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:00.174075 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:00.174078 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:00.174080 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:00.174158 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:00.174485 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:00.174488 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:00.174575 (+ 87us) client_negotiation.cc:769] Sending connection context
0808 08:25:00.174658 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 08:25:00.174787 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":107,"threads_started":1}
I20260808 08:25:00.175004 25273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:00.167969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:45973 (local address 127.22.235.254:46093)
0808 08:25:00.168086 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:00.168089 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:00.168224 (+ 135us) server_negotiation.cc:407] Connection header received
0808 08:25:00.168387 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:00.168389 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:00.168433 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:00.168511 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:00.168972 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.169700 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.170692 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.170944 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.171041 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:00.172086 (+ 1045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:00.172089 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:00.172091 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:00.172113 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:00.173483 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:00.173915 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:00.173919 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:00.173921 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:00.173995 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:00.174236 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:00.174240 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:00.174241 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:00.174399 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:00.174481 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:00.174719 (+ 238us) server_negotiation.cc:299] Negotiation successful
0808 08:25:00.174817 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
Aug 08 08:25:00 dist-test-slave-f7th krb5kdc[25125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177500, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 08:25:00 dist-test-slave-f7th krb5kdc[25125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177500, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:25:00.197441 25273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:00.190121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33410 (local address 127.22.235.254:46093)
0808 08:25:00.190236 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:00.190239 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:00.190250 (+ 11us) server_negotiation.cc:407] Connection header received
0808 08:25:00.190284 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:00.190286 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:00.190322 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:00.190386 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:00.191044 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.191550 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.192336 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.192551 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.192653 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:00.194764 (+ 2111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:00.194767 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:00.194769 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:00.194791 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:00.195835 (+ 1044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:00.196212 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:00.196215 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:00.196216 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:00.196287 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:00.196588 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:00.196591 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:00.196593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:00.196793 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:00.196913 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:00.197177 (+ 264us) server_negotiation.cc:299] Negotiation successful
0808 08:25:00.197284 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 08:25:00.200040 25188 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33410:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 08:25:00.203068 25188 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 08:25:00.216174 26019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:00.211008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:42107 (local address 127.0.0.1:41306)
0808 08:25:00.211352 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:00.211402 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:00.211543 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:00.212379 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:00.212385 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:00.212412 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:25:00.212543 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:00.212551 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.213808 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.213811 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:00.214853 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:00.214862 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.215676 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.215680 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:00.215783 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:00.215822 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 08:25:00.215918 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 08:25:00.216060 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":90,"threads_started":1}
I20260808 08:25:00.216176 26020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:00.211123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41306 (local address 127.22.235.193:42107)
0808 08:25:00.211841 (+ 718us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:00.211856 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:00.211881 (+ 25us) server_negotiation.cc:407] Connection header received
0808 08:25:00.211961 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:00.211966 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:00.212181 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:00.212388 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:25:00.212700 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.213715 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.214986 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.215564 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.215692 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:00.215887 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:00.215941 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 08:25:00.216028 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":91,"threads_started":1}
I20260808 08:25:00.217334 26022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:00.211719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.195:39991 (local address 127.0.0.1:56588)
0808 08:25:00.212672 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:00.212683 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:00.212774 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:00.213157 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:00.213160 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:00.213178 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:25:00.213277 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:00.213284 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.214609 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.214612 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:00.215847 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:00.215859 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.216946 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.216950 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:00.217038 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:00.217062 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 08:25:00.217128 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 08:25:00.217206 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":865,"mutex_wait_us":23,"thread_start_us":86,"threads_started":1}
I20260808 08:25:00.217382 26029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:00.212137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56588 (local address 127.22.235.195:39991)
0808 08:25:00.212561 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:00.212572 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:00.212760 (+ 188us) server_negotiation.cc:407] Connection header received
0808 08:25:00.212881 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:00.212886 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:00.213039 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:00.213167 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:25:00.213381 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.214478 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.216027 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.216827 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.216962 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:00.217094 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:00.217151 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 08:25:00.217238 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":113,"threads_started":1}
I20260808 08:25:00.218132 26026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:00.211356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.194:44277 (local address 127.0.0.1:33162)
0808 08:25:00.212507 (+ 1151us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:00.212521 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:00.212699 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:00.213291 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:00.213293 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:00.213311 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:25:00.213407 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:00.213412 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.214716 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.214719 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:00.215881 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:00.215888 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.217738 (+ 1850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.217742 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:00.217841 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:00.217864 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 08:25:00.217938 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 08:25:00.218009 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1081,"mutex_wait_us":17,"thread_start_us":106,"threads_started":1}
I20260808 08:25:00.218231 26025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:00.211636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33162 (local address 127.22.235.194:44277)
0808 08:25:00.212204 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:00.212217 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:00.212606 (+ 389us) server_negotiation.cc:407] Connection header received
0808 08:25:00.212841 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:00.212846 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:00.213019 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:00.213136 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:25:00.213528 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.214606 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.216006 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.217165 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.217291 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:00.217674 (+ 383us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:00.218005 (+ 331us) server_negotiation.cc:299] Negotiation successful
0808 08:25:00.218098 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":113,"threads_started":1}
I20260808 08:25:00.219187 25412 tablet_service.cc:1467] Processing CreateTablet for tablet dada9ec412d84d749d96321db19d36d3 (DEFAULT_TABLE table=test-table [id=53218eb8f9194f3f9ff50318def24038]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:25:00.220124 25412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dada9ec412d84d749d96321db19d36d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:00.221640 25883 tablet_service.cc:1467] Processing CreateTablet for tablet dada9ec412d84d749d96321db19d36d3 (DEFAULT_TABLE table=test-table [id=53218eb8f9194f3f9ff50318def24038]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:25:00.222543 25883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dada9ec412d84d749d96321db19d36d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:00.222267 25651 tablet_service.cc:1467] Processing CreateTablet for tablet dada9ec412d84d749d96321db19d36d3 (DEFAULT_TABLE table=test-table [id=53218eb8f9194f3f9ff50318def24038]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:25:00.223259 25651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dada9ec412d84d749d96321db19d36d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:00.228578 26031 tablet_bootstrap.cc:492] T dada9ec412d84d749d96321db19d36d3 P 8a07e1d1baaf41689220531bd2692f11: Bootstrap starting.
I20260808 08:25:00.229550 26032 tablet_bootstrap.cc:492] T dada9ec412d84d749d96321db19d36d3 P 60a28860872c4a078b71afee9b53ace1: Bootstrap starting.
I20260808 08:25:00.230808 26033 tablet_bootstrap.cc:492] T dada9ec412d84d749d96321db19d36d3 P 15380f70591f440a810a5e6e46fa89ad: Bootstrap starting.
I20260808 08:25:00.231817 26032 tablet_bootstrap.cc:654] T dada9ec412d84d749d96321db19d36d3 P 60a28860872c4a078b71afee9b53ace1: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:00.231817 26031 tablet_bootstrap.cc:654] T dada9ec412d84d749d96321db19d36d3 P 8a07e1d1baaf41689220531bd2692f11: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:00.232457 26033 tablet_bootstrap.cc:654] T dada9ec412d84d749d96321db19d36d3 P 15380f70591f440a810a5e6e46fa89ad: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:00.232728 26032 log.cc:824] T dada9ec412d84d749d96321db19d36d3 P 60a28860872c4a078b71afee9b53ace1: Log is configured to *not* fsync() on all Append() calls
I20260808 08:25:00.232728 26031 log.cc:824] T dada9ec412d84d749d96321db19d36d3 P 8a07e1d1baaf41689220531bd2692f11: Log is configured to *not* fsync() on all Append() calls
I20260808 08:25:00.233217 26033 log.cc:824] T dada9ec412d84d749d96321db19d36d3 P 15380f70591f440a810a5e6e46fa89ad: Log is configured to *not* fsync() on all Append() calls
I20260808 08:25:00.234666 26032 tablet_bootstrap.cc:492] T dada9ec412d84d749d96321db19d36d3 P 60a28860872c4a078b71afee9b53ace1: No bootstrap required, opened a new log
I20260808 08:25:00.234668 26031 tablet_bootstrap.cc:492] T dada9ec412d84d749d96321db19d36d3 P 8a07e1d1baaf41689220531bd2692f11: No bootstrap required, opened a new log
I20260808 08:25:00.234858 26031 ts_tablet_manager.cc:1397] T dada9ec412d84d749d96321db19d36d3 P 8a07e1d1baaf41689220531bd2692f11: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260808 08:25:00.234985 26033 tablet_bootstrap.cc:492] T dada9ec412d84d749d96321db19d36d3 P 15380f70591f440a810a5e6e46fa89ad: No bootstrap required, opened a new log
I20260808 08:25:00.234977 26032 ts_tablet_manager.cc:1397] T dada9ec412d84d749d96321db19d36d3 P 60a28860872c4a078b71afee9b53ace1: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 08:25:00.235148 26033 ts_tablet_manager.cc:1397] T dada9ec412d84d749d96321db19d36d3 P 15380f70591f440a810a5e6e46fa89ad: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260808 08:25:00.238139 26032 raft_consensus.cc:348] T dada9ec412d84d749d96321db19d36d3 P 60a28860872c4a078b71afee9b53ace1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a07e1d1baaf41689220531bd2692f11" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 42107 } } peers { permanent_uuid: "60a28860872c4a078b71afee9b53ace1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39991 } } peers { permanent_uuid: "15380f70591f440a810a5e6e46fa89ad" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44277 } }
I20260808 08:25:00.238545 26032 raft_consensus.cc:374] T dada9ec412d84d749d96321db19d36d3 P 60a28860872c4a078b71afee9b53ace1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:00.238616 26032 raft_consensus.cc:729] T dada9ec412d84d749d96321db19d36d3 P 60a28860872c4a078b71afee9b53ace1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60a28860872c4a078b71afee9b53ace1, State: Initialized, Role: FOLLOWER
I20260808 08:25:00.238646 26033 raft_consensus.cc:348] T dada9ec412d84d749d96321db19d36d3 P 15380f70591f440a810a5e6e46fa89ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a07e1d1baaf41689220531bd2692f11" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 42107 } } peers { permanent_uuid: "60a28860872c4a078b71afee9b53ace1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39991 } } peers { permanent_uuid: "15380f70591f440a810a5e6e46fa89ad" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44277 } }
I20260808 08:25:00.238646 26031 raft_consensus.cc:348] T dada9ec412d84d749d96321db19d36d3 P 8a07e1d1baaf41689220531bd2692f11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a07e1d1baaf41689220531bd2692f11" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 42107 } } peers { permanent_uuid: "60a28860872c4a078b71afee9b53ace1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39991 } } peers { permanent_uuid: "15380f70591f440a810a5e6e46fa89ad" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44277 } }
I20260808 08:25:00.238883 26033 raft_consensus.cc:374] T dada9ec412d84d749d96321db19d36d3 P 15380f70591f440a810a5e6e46fa89ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:00.238883 26031 raft_consensus.cc:374] T dada9ec412d84d749d96321db19d36d3 P 8a07e1d1baaf41689220531bd2692f11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:00.238942 26033 raft_consensus.cc:729] T dada9ec412d84d749d96321db19d36d3 P 15380f70591f440a810a5e6e46fa89ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15380f70591f440a810a5e6e46fa89ad, State: Initialized, Role: FOLLOWER
I20260808 08:25:00.238942 26031 raft_consensus.cc:729] T dada9ec412d84d749d96321db19d36d3 P 8a07e1d1baaf41689220531bd2692f11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a07e1d1baaf41689220531bd2692f11, State: Initialized, Role: FOLLOWER
I20260808 08:25:00.239074 26032 consensus_queue.cc:260] T dada9ec412d84d749d96321db19d36d3 P 60a28860872c4a078b71afee9b53ace1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a07e1d1baaf41689220531bd2692f11" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 42107 } } peers { permanent_uuid: "60a28860872c4a078b71afee9b53ace1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39991 } } peers { permanent_uuid: "15380f70591f440a810a5e6e46fa89ad" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44277 } }
I20260808 08:25:00.239356 26033 consensus_queue.cc:260] T dada9ec412d84d749d96321db19d36d3 P 15380f70591f440a810a5e6e46fa89ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a07e1d1baaf41689220531bd2692f11" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 42107 } } peers { permanent_uuid: "60a28860872c4a078b71afee9b53ace1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39991 } } peers { permanent_uuid: "15380f70591f440a810a5e6e46fa89ad" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44277 } }
I20260808 08:25:00.239356 26031 consensus_queue.cc:260] T dada9ec412d84d749d96321db19d36d3 P 8a07e1d1baaf41689220531bd2692f11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a07e1d1baaf41689220531bd2692f11" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 42107 } } peers { permanent_uuid: "60a28860872c4a078b71afee9b53ace1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39991 } } peers { permanent_uuid: "15380f70591f440a810a5e6e46fa89ad" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44277 } }
I20260808 08:25:00.239764 25995 heartbeater.cc:503] Master 127.22.235.254:46093 was elected leader, sending a full tablet report...
I20260808 08:25:00.240058 26032 ts_tablet_manager.cc:1428] T dada9ec412d84d749d96321db19d36d3 P 60a28860872c4a078b71afee9b53ace1: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260808 08:25:00.240252 26031 ts_tablet_manager.cc:1428] T dada9ec412d84d749d96321db19d36d3 P 8a07e1d1baaf41689220531bd2692f11: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260808 08:25:00.240252 26033 ts_tablet_manager.cc:1428] T dada9ec412d84d749d96321db19d36d3 P 15380f70591f440a810a5e6e46fa89ad: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 08:25:00.240564 25518 heartbeater.cc:503] Master 127.22.235.254:46093 was elected leader, sending a full tablet report...
I20260808 08:25:00.240564 25754 heartbeater.cc:503] Master 127.22.235.254:46093 was elected leader, sending a full tablet report...
I20260808 08:25:00.352489 26038 raft_consensus.cc:482] T dada9ec412d84d749d96321db19d36d3 P 60a28860872c4a078b71afee9b53ace1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 08:25:00.352746 26038 raft_consensus.cc:504] T dada9ec412d84d749d96321db19d36d3 P 60a28860872c4a078b71afee9b53ace1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a07e1d1baaf41689220531bd2692f11" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 42107 } } peers { permanent_uuid: "60a28860872c4a078b71afee9b53ace1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39991 } } peers { permanent_uuid: "15380f70591f440a810a5e6e46fa89ad" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44277 } }
I20260808 08:25:00.353929 26038 leader_election.cc:288] T dada9ec412d84d749d96321db19d36d3 P 60a28860872c4a078b71afee9b53ace1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a07e1d1baaf41689220531bd2692f11 (127.22.235.193:42107), 15380f70591f440a810a5e6e46fa89ad (127.22.235.194:44277)
I20260808 08:25:00.358009 26020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:00.354295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:48717 (local address 127.22.235.193:42107)
0808 08:25:00.354423 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:00.354428 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:00.354478 (+ 50us) server_negotiation.cc:407] Connection header received
0808 08:25:00.354651 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:00.354654 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:00.354708 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:00.354785 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:25:00.355090 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.355686 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.356880 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.357500 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.357615 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:00.357672 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:00.357849 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 08:25:00.357898 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 08:25:00.358048 25811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:00.354189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:42107 (local address 127.22.235.195:48717)
0808 08:25:00.354412 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:00.354426 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:00.354533 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:00.354814 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:00.354817 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:00.354836 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:25:00.354982 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:00.354990 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.355788 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.355792 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:00.356761 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:00.356771 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.357606 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.357610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:00.357729 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:00.357758 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 08:25:00.357836 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 08:25:00.357904 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143}
I20260808 08:25:00.358726 26025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:00.354452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:38017 (local address 127.22.235.194:44277)
0808 08:25:00.354631 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:00.354636 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:00.354781 (+ 145us) server_negotiation.cc:407] Connection header received
0808 08:25:00.354902 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:00.354905 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:00.354973 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:00.355051 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:25:00.355316 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.355870 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.357661 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.358201 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.358314 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:00.358384 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:00.358554 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 08:25:00.358624 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260808 08:25:00.358727 26043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:00.354192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.194:44277 (local address 127.22.235.195:38017)
0808 08:25:00.354695 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:00.354709 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:00.354808 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:00.355073 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:00.355076 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:00.355094 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:25:00.355218 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:00.355224 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.355997 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.356001 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:00.357541 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:00.357552 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.358322 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.358325 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:00.358424 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:00.358448 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 08:25:00.358521 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 08:25:00.358587 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"mutex_wait_us":46,"thread_start_us":151,"threads_started":1}
I20260808 08:25:00.358702 25440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dada9ec412d84d749d96321db19d36d3" candidate_uuid: "60a28860872c4a078b71afee9b53ace1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a07e1d1baaf41689220531bd2692f11" is_pre_election: true
I20260808 08:25:00.359058 25440 raft_consensus.cc:2454] T dada9ec412d84d749d96321db19d36d3 P 8a07e1d1baaf41689220531bd2692f11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60a28860872c4a078b71afee9b53ace1 in term 0.
I20260808 08:25:00.359370 25680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dada9ec412d84d749d96321db19d36d3" candidate_uuid: "60a28860872c4a078b71afee9b53ace1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15380f70591f440a810a5e6e46fa89ad" is_pre_election: true
I20260808 08:25:00.359576 25801 leader_election.cc:302] T dada9ec412d84d749d96321db19d36d3 P 60a28860872c4a078b71afee9b53ace1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60a28860872c4a078b71afee9b53ace1, 8a07e1d1baaf41689220531bd2692f11; no voters:
I20260808 08:25:00.359658 25680 raft_consensus.cc:2454] T dada9ec412d84d749d96321db19d36d3 P 15380f70591f440a810a5e6e46fa89ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60a28860872c4a078b71afee9b53ace1 in term 0.
I20260808 08:25:00.359833 26038 raft_consensus.cc:2781] T dada9ec412d84d749d96321db19d36d3 P 60a28860872c4a078b71afee9b53ace1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 08:25:00.359930 26038 raft_consensus.cc:482] T dada9ec412d84d749d96321db19d36d3 P 60a28860872c4a078b71afee9b53ace1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 08:25:00.359984 26038 raft_consensus.cc:3037] T dada9ec412d84d749d96321db19d36d3 P 60a28860872c4a078b71afee9b53ace1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:00.361268 26038 raft_consensus.cc:504] T dada9ec412d84d749d96321db19d36d3 P 60a28860872c4a078b71afee9b53ace1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a07e1d1baaf41689220531bd2692f11" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 42107 } } peers { permanent_uuid: "60a28860872c4a078b71afee9b53ace1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39991 } } peers { permanent_uuid: "15380f70591f440a810a5e6e46fa89ad" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44277 } }
I20260808 08:25:00.361572 26038 leader_election.cc:288] T dada9ec412d84d749d96321db19d36d3 P 60a28860872c4a078b71afee9b53ace1 [CANDIDATE]: Term 1 election: Requested vote from peers 8a07e1d1baaf41689220531bd2692f11 (127.22.235.193:42107), 15380f70591f440a810a5e6e46fa89ad (127.22.235.194:44277)
I20260808 08:25:00.361874 25440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dada9ec412d84d749d96321db19d36d3" candidate_uuid: "60a28860872c4a078b71afee9b53ace1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a07e1d1baaf41689220531bd2692f11"
I20260808 08:25:00.361910 25680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dada9ec412d84d749d96321db19d36d3" candidate_uuid: "60a28860872c4a078b71afee9b53ace1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15380f70591f440a810a5e6e46fa89ad"
I20260808 08:25:00.361984 25440 raft_consensus.cc:3037] T dada9ec412d84d749d96321db19d36d3 P 8a07e1d1baaf41689220531bd2692f11 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:00.362046 25680 raft_consensus.cc:3037] T dada9ec412d84d749d96321db19d36d3 P 15380f70591f440a810a5e6e46fa89ad [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:00.362939 25440 raft_consensus.cc:2454] T dada9ec412d84d749d96321db19d36d3 P 8a07e1d1baaf41689220531bd2692f11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60a28860872c4a078b71afee9b53ace1 in term 1.
I20260808 08:25:00.363216 25801 leader_election.cc:302] T dada9ec412d84d749d96321db19d36d3 P 60a28860872c4a078b71afee9b53ace1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60a28860872c4a078b71afee9b53ace1, 8a07e1d1baaf41689220531bd2692f11; no voters:
I20260808 08:25:00.363386 26038 raft_consensus.cc:2781] T dada9ec412d84d749d96321db19d36d3 P 60a28860872c4a078b71afee9b53ace1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:25:00.363515 25680 raft_consensus.cc:2454] T dada9ec412d84d749d96321db19d36d3 P 15380f70591f440a810a5e6e46fa89ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60a28860872c4a078b71afee9b53ace1 in term 1.
I20260808 08:25:00.363582 26038 raft_consensus.cc:686] T dada9ec412d84d749d96321db19d36d3 P 60a28860872c4a078b71afee9b53ace1 [term 1 LEADER]: Becoming Leader. State: Replica: 60a28860872c4a078b71afee9b53ace1, State: Running, Role: LEADER
I20260808 08:25:00.364018 26038 consensus_queue.cc:237] T dada9ec412d84d749d96321db19d36d3 P 60a28860872c4a078b71afee9b53ace1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a07e1d1baaf41689220531bd2692f11" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 42107 } } peers { permanent_uuid: "60a28860872c4a078b71afee9b53ace1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39991 } } peers { permanent_uuid: "15380f70591f440a810a5e6e46fa89ad" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44277 } }
I20260808 08:25:00.367208 25190 catalog_manager.cc:5495] T dada9ec412d84d749d96321db19d36d3 P 60a28860872c4a078b71afee9b53ace1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 60a28860872c4a078b71afee9b53ace1 (127.22.235.195). New cstate: current_term: 1 leader_uuid: "60a28860872c4a078b71afee9b53ace1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a07e1d1baaf41689220531bd2692f11" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 42107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "60a28860872c4a078b71afee9b53ace1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15380f70591f440a810a5e6e46fa89ad" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44277 } health_report { overall_health: UNKNOWN } } }
I20260808 08:25:00.411216 26029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:00.407513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56600 (local address 127.22.235.195:39991)
0808 08:25:00.407665 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:00.407670 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:00.407685 (+ 15us) server_negotiation.cc:407] Connection header received
0808 08:25:00.407866 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:00.407868 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:00.407933 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:00.408025 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:25:00.408303 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.408926 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.409785 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.410017 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.410105 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:00.410420 (+ 315us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:25:00.410813 (+ 393us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 08:25:00.410916 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:00.411024 (+ 108us) server_negotiation.cc:299] Negotiation successful
0808 08:25:00.411081 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260808 08:25:00.412554 25996 tablet.cc:2369] T dada9ec412d84d749d96321db19d36d3 P 60a28860872c4a078b71afee9b53ace1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 08:25:00.416775 25440 raft_consensus.cc:1261] T dada9ec412d84d749d96321db19d36d3 P 8a07e1d1baaf41689220531bd2692f11 [term 1 FOLLOWER]: Refusing update from remote peer 60a28860872c4a078b71afee9b53ace1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:25:00.416775 25680 raft_consensus.cc:1261] T dada9ec412d84d749d96321db19d36d3 P 15380f70591f440a810a5e6e46fa89ad [term 1 FOLLOWER]: Refusing update from remote peer 60a28860872c4a078b71afee9b53ace1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:25:00.417475 26047 consensus_queue.cc:1035] T dada9ec412d84d749d96321db19d36d3 P 60a28860872c4a078b71afee9b53ace1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a07e1d1baaf41689220531bd2692f11" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 42107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:25:00.417636 26038 consensus_queue.cc:1035] T dada9ec412d84d749d96321db19d36d3 P 60a28860872c4a078b71afee9b53ace1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15380f70591f440a810a5e6e46fa89ad" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:25:00.436403 25188 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33410:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:25:00.436713 25188 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33410:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:25:00.439613 25188 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f3651710b85e4fe9be5fd28aff89bf89: Sending DeleteTablet for 3 replicas of tablet dada9ec412d84d749d96321db19d36d3
I20260808 08:25:00.440382 25651 tablet_service.cc:1514] Processing DeleteTablet for tablet dada9ec412d84d749d96321db19d36d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:25:00 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:33162
I20260808 08:25:00.440464 25412 tablet_service.cc:1514] Processing DeleteTablet for tablet dada9ec412d84d749d96321db19d36d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:25:00 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:41306
I20260808 08:25:00.441145 25883 tablet_service.cc:1514] Processing DeleteTablet for tablet dada9ec412d84d749d96321db19d36d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:25:00 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:56588
I20260808 08:25:00.441514 26068 tablet_replica.cc:330] T dada9ec412d84d749d96321db19d36d3 P 8a07e1d1baaf41689220531bd2692f11: stopping tablet replica
I20260808 08:25:00.442022 26068 raft_consensus.cc:2229] T dada9ec412d84d749d96321db19d36d3 P 8a07e1d1baaf41689220531bd2692f11 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 08:25:00.442266 26067 tablet_replica.cc:330] T dada9ec412d84d749d96321db19d36d3 P 15380f70591f440a810a5e6e46fa89ad: stopping tablet replica
I20260808 08:25:00.442373 26068 raft_consensus.cc:2258] T dada9ec412d84d749d96321db19d36d3 P 8a07e1d1baaf41689220531bd2692f11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:25:00.442250 25188 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33410:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 08:25:00.442644 26067 raft_consensus.cc:2229] T dada9ec412d84d749d96321db19d36d3 P 15380f70591f440a810a5e6e46fa89ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 08:25:00.442937 26067 raft_consensus.cc:2258] T dada9ec412d84d749d96321db19d36d3 P 15380f70591f440a810a5e6e46fa89ad [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 08:25:00.443146 25188 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 08:25:00.444468 26068 ts_tablet_manager.cc:1905] T dada9ec412d84d749d96321db19d36d3 P 8a07e1d1baaf41689220531bd2692f11: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:25:00.447810 26068 ts_tablet_manager.cc:1918] T dada9ec412d84d749d96321db19d36d3 P 8a07e1d1baaf41689220531bd2692f11: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 08:25:00.447911 26068 log.cc:1196] T dada9ec412d84d749d96321db19d36d3 P 8a07e1d1baaf41689220531bd2692f11: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-0/wal/wals/dada9ec412d84d749d96321db19d36d3
I20260808 08:25:00.448225 26068 ts_tablet_manager.cc:1939] T dada9ec412d84d749d96321db19d36d3 P 8a07e1d1baaf41689220531bd2692f11: Deleting consensus metadata
I20260808 08:25:00.448204 26067 ts_tablet_manager.cc:1905] T dada9ec412d84d749d96321db19d36d3 P 15380f70591f440a810a5e6e46fa89ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:25:00.448369 25412 tablet_service.cc:1467] Processing CreateTablet for tablet 1addfb5834924e36bd36f6175c33c3c9 (DEFAULT_TABLE table=test-table [id=a3249cf243674e13be890b250d2856ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:25:00.448635 25412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1addfb5834924e36bd36f6175c33c3c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:00.449473 25164 catalog_manager.cc:4841] TS 8a07e1d1baaf41689220531bd2692f11 (127.22.235.193:42107): tablet dada9ec412d84d749d96321db19d36d3 (table test-table [id=53218eb8f9194f3f9ff50318def24038]) successfully deleted
I20260808 08:25:00.449712 25883 tablet_service.cc:1467] Processing CreateTablet for tablet 1addfb5834924e36bd36f6175c33c3c9 (DEFAULT_TABLE table=test-table [id=a3249cf243674e13be890b250d2856ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:25:00.449971 25883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1addfb5834924e36bd36f6175c33c3c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:00.450606 26067 ts_tablet_manager.cc:1918] T dada9ec412d84d749d96321db19d36d3 P 15380f70591f440a810a5e6e46fa89ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 08:25:00.450686 26067 log.cc:1196] T dada9ec412d84d749d96321db19d36d3 P 15380f70591f440a810a5e6e46fa89ad: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-1/wal/wals/dada9ec412d84d749d96321db19d36d3
I20260808 08:25:00.450730 26031 tablet_bootstrap.cc:492] T 1addfb5834924e36bd36f6175c33c3c9 P 8a07e1d1baaf41689220531bd2692f11: Bootstrap starting.
I20260808 08:25:00.450989 26067 ts_tablet_manager.cc:1939] T dada9ec412d84d749d96321db19d36d3 P 15380f70591f440a810a5e6e46fa89ad: Deleting consensus metadata
I20260808 08:25:00.451642 26031 tablet_bootstrap.cc:654] T 1addfb5834924e36bd36f6175c33c3c9 P 8a07e1d1baaf41689220531bd2692f11: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:00.452270 26032 tablet_bootstrap.cc:492] T 1addfb5834924e36bd36f6175c33c3c9 P 60a28860872c4a078b71afee9b53ace1: Bootstrap starting.
I20260808 08:25:00.452451 25165 catalog_manager.cc:4841] TS 15380f70591f440a810a5e6e46fa89ad (127.22.235.194:44277): tablet dada9ec412d84d749d96321db19d36d3 (table test-table [id=53218eb8f9194f3f9ff50318def24038]) successfully deleted
I20260808 08:25:00.452780 25651 tablet_service.cc:1467] Processing CreateTablet for tablet 1addfb5834924e36bd36f6175c33c3c9 (DEFAULT_TABLE table=test-table [id=a3249cf243674e13be890b250d2856ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:25:00.453028 25651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1addfb5834924e36bd36f6175c33c3c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:00.453442 26032 tablet_bootstrap.cc:654] T 1addfb5834924e36bd36f6175c33c3c9 P 60a28860872c4a078b71afee9b53ace1: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:00.454043 26071 tablet_replica.cc:330] T dada9ec412d84d749d96321db19d36d3 P 60a28860872c4a078b71afee9b53ace1: stopping tablet replica
I20260808 08:25:00.454313 26071 raft_consensus.cc:2229] T dada9ec412d84d749d96321db19d36d3 P 60a28860872c4a078b71afee9b53ace1 [term 1 LEADER]: Raft consensus shutting down.
I20260808 08:25:00.454656 26071 raft_consensus.cc:2258] T dada9ec412d84d749d96321db19d36d3 P 60a28860872c4a078b71afee9b53ace1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:25:00.455072 26033 tablet_bootstrap.cc:492] T 1addfb5834924e36bd36f6175c33c3c9 P 15380f70591f440a810a5e6e46fa89ad: Bootstrap starting.
I20260808 08:25:00.455847 26032 tablet_bootstrap.cc:492] T 1addfb5834924e36bd36f6175c33c3c9 P 60a28860872c4a078b71afee9b53ace1: No bootstrap required, opened a new log
I20260808 08:25:00.455864 26031 tablet_bootstrap.cc:492] T 1addfb5834924e36bd36f6175c33c3c9 P 8a07e1d1baaf41689220531bd2692f11: No bootstrap required, opened a new log
I20260808 08:25:00.455924 26032 ts_tablet_manager.cc:1397] T 1addfb5834924e36bd36f6175c33c3c9 P 60a28860872c4a078b71afee9b53ace1: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260808 08:25:00.455941 26031 ts_tablet_manager.cc:1397] T 1addfb5834924e36bd36f6175c33c3c9 P 8a07e1d1baaf41689220531bd2692f11: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20260808 08:25:00.455984 26033 tablet_bootstrap.cc:654] T 1addfb5834924e36bd36f6175c33c3c9 P 15380f70591f440a810a5e6e46fa89ad: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:00.456385 26031 raft_consensus.cc:348] T 1addfb5834924e36bd36f6175c33c3c9 P 8a07e1d1baaf41689220531bd2692f11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a07e1d1baaf41689220531bd2692f11" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 42107 } } peers { permanent_uuid: "15380f70591f440a810a5e6e46fa89ad" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44277 } } peers { permanent_uuid: "60a28860872c4a078b71afee9b53ace1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39991 } }
I20260808 08:25:00.456388 26032 raft_consensus.cc:348] T 1addfb5834924e36bd36f6175c33c3c9 P 60a28860872c4a078b71afee9b53ace1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a07e1d1baaf41689220531bd2692f11" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 42107 } } peers { permanent_uuid: "15380f70591f440a810a5e6e46fa89ad" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44277 } } peers { permanent_uuid: "60a28860872c4a078b71afee9b53ace1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39991 } }
I20260808 08:25:00.456488 26032 raft_consensus.cc:374] T 1addfb5834924e36bd36f6175c33c3c9 P 60a28860872c4a078b71afee9b53ace1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:00.456517 26032 raft_consensus.cc:729] T 1addfb5834924e36bd36f6175c33c3c9 P 60a28860872c4a078b71afee9b53ace1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60a28860872c4a078b71afee9b53ace1, State: Initialized, Role: FOLLOWER
I20260808 08:25:00.456512 26031 raft_consensus.cc:374] T 1addfb5834924e36bd36f6175c33c3c9 P 8a07e1d1baaf41689220531bd2692f11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:00.456555 26031 raft_consensus.cc:729] T 1addfb5834924e36bd36f6175c33c3c9 P 8a07e1d1baaf41689220531bd2692f11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a07e1d1baaf41689220531bd2692f11, State: Initialized, Role: FOLLOWER
I20260808 08:25:00.456727 26031 consensus_queue.cc:260] T 1addfb5834924e36bd36f6175c33c3c9 P 8a07e1d1baaf41689220531bd2692f11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a07e1d1baaf41689220531bd2692f11" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 42107 } } peers { permanent_uuid: "15380f70591f440a810a5e6e46fa89ad" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44277 } } peers { permanent_uuid: "60a28860872c4a078b71afee9b53ace1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39991 } }
I20260808 08:25:00.456794 26032 consensus_queue.cc:260] T 1addfb5834924e36bd36f6175c33c3c9 P 60a28860872c4a078b71afee9b53ace1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a07e1d1baaf41689220531bd2692f11" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 42107 } } peers { permanent_uuid: "15380f70591f440a810a5e6e46fa89ad" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44277 } } peers { permanent_uuid: "60a28860872c4a078b71afee9b53ace1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39991 } }
I20260808 08:25:00.456981 26031 ts_tablet_manager.cc:1428] T 1addfb5834924e36bd36f6175c33c3c9 P 8a07e1d1baaf41689220531bd2692f11: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 08:25:00.457558 26032 ts_tablet_manager.cc:1428] T 1addfb5834924e36bd36f6175c33c3c9 P 60a28860872c4a078b71afee9b53ace1: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260808 08:25:00.459074 26071 ts_tablet_manager.cc:1905] T dada9ec412d84d749d96321db19d36d3 P 60a28860872c4a078b71afee9b53ace1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:25:00.461077 26033 tablet_bootstrap.cc:492] T 1addfb5834924e36bd36f6175c33c3c9 P 15380f70591f440a810a5e6e46fa89ad: No bootstrap required, opened a new log
I20260808 08:25:00.461170 26033 ts_tablet_manager.cc:1397] T 1addfb5834924e36bd36f6175c33c3c9 P 15380f70591f440a810a5e6e46fa89ad: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260808 08:25:00.461531 26071 ts_tablet_manager.cc:1918] T dada9ec412d84d749d96321db19d36d3 P 60a28860872c4a078b71afee9b53ace1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 08:25:00.461614 26071 log.cc:1196] T dada9ec412d84d749d96321db19d36d3 P 60a28860872c4a078b71afee9b53ace1: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-2/wal/wals/dada9ec412d84d749d96321db19d36d3
I20260808 08:25:00.461606 26033 raft_consensus.cc:348] T 1addfb5834924e36bd36f6175c33c3c9 P 15380f70591f440a810a5e6e46fa89ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a07e1d1baaf41689220531bd2692f11" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 42107 } } peers { permanent_uuid: "15380f70591f440a810a5e6e46fa89ad" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44277 } } peers { permanent_uuid: "60a28860872c4a078b71afee9b53ace1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39991 } }
I20260808 08:25:00.461726 26033 raft_consensus.cc:374] T 1addfb5834924e36bd36f6175c33c3c9 P 15380f70591f440a810a5e6e46fa89ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:00.461766 26033 raft_consensus.cc:729] T 1addfb5834924e36bd36f6175c33c3c9 P 15380f70591f440a810a5e6e46fa89ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15380f70591f440a810a5e6e46fa89ad, State: Initialized, Role: FOLLOWER
I20260808 08:25:00.461935 26071 ts_tablet_manager.cc:1939] T dada9ec412d84d749d96321db19d36d3 P 60a28860872c4a078b71afee9b53ace1: Deleting consensus metadata
I20260808 08:25:00.461894 26033 consensus_queue.cc:260] T 1addfb5834924e36bd36f6175c33c3c9 P 15380f70591f440a810a5e6e46fa89ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a07e1d1baaf41689220531bd2692f11" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 42107 } } peers { permanent_uuid: "15380f70591f440a810a5e6e46fa89ad" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44277 } } peers { permanent_uuid: "60a28860872c4a078b71afee9b53ace1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39991 } }
I20260808 08:25:00.462172 26033 ts_tablet_manager.cc:1428] T 1addfb5834924e36bd36f6175c33c3c9 P 15380f70591f440a810a5e6e46fa89ad: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 08:25:00.463064 25164 catalog_manager.cc:4841] TS 60a28860872c4a078b71afee9b53ace1 (127.22.235.195:39991): tablet dada9ec412d84d749d96321db19d36d3 (table test-table [id=53218eb8f9194f3f9ff50318def24038]) successfully deleted
I20260808 08:25:00.474220 26040 raft_consensus.cc:482] T 1addfb5834924e36bd36f6175c33c3c9 P 8a07e1d1baaf41689220531bd2692f11 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 08:25:00.474326 26040 raft_consensus.cc:504] T 1addfb5834924e36bd36f6175c33c3c9 P 8a07e1d1baaf41689220531bd2692f11 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a07e1d1baaf41689220531bd2692f11" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 42107 } } peers { permanent_uuid: "15380f70591f440a810a5e6e46fa89ad" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44277 } } peers { permanent_uuid: "60a28860872c4a078b71afee9b53ace1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39991 } }
I20260808 08:25:00.475829 26040 leader_election.cc:288] T 1addfb5834924e36bd36f6175c33c3c9 P 8a07e1d1baaf41689220531bd2692f11 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15380f70591f440a810a5e6e46fa89ad (127.22.235.194:44277), 60a28860872c4a078b71afee9b53ace1 (127.22.235.195:39991)
I20260808 08:25:00.480422 26029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:00.476442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:50547 (local address 127.22.235.195:39991)
0808 08:25:00.476554 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:00.476558 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:00.476589 (+ 31us) server_negotiation.cc:407] Connection header received
0808 08:25:00.476758 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:00.476761 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:00.476802 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:00.476871 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:25:00.477205 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.477960 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.479175 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.479725 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.479829 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:00.479888 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:00.480237 (+ 349us) server_negotiation.cc:299] Negotiation successful
0808 08:25:00.480309 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 08:25:00.480450 26078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:00.476067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.195:39991 (local address 127.22.235.193:50547)
0808 08:25:00.476517 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:00.476531 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:00.476660 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:00.476888 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:00.476891 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:00.476936 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:25:00.477107 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:00.477113 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.478102 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.478106 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:00.479062 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:00.479072 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.480017 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.480021 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:00.480119 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:00.480146 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 08:25:00.480221 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 08:25:00.480286 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"mutex_wait_us":70,"spinlock_wait_cycles":39680,"thread_start_us":100,"threads_started":1}
I20260808 08:25:00.480914 26025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:00.476136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:58701 (local address 127.22.235.194:44277)
0808 08:25:00.476264 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:00.476269 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:00.476388 (+ 119us) server_negotiation.cc:407] Connection header received
0808 08:25:00.476599 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:00.476602 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:00.476701 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:00.476751 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:25:00.477173 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.477946 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.479609 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.480227 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.480337 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:00.480386 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:00.480722 (+ 336us) server_negotiation.cc:299] Negotiation successful
0808 08:25:00.480803 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 08:25:00.480998 25921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1addfb5834924e36bd36f6175c33c3c9" candidate_uuid: "8a07e1d1baaf41689220531bd2692f11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60a28860872c4a078b71afee9b53ace1" is_pre_election: true
I20260808 08:25:00.481227 25921 raft_consensus.cc:2454] T 1addfb5834924e36bd36f6175c33c3c9 P 60a28860872c4a078b71afee9b53ace1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a07e1d1baaf41689220531bd2692f11 in term 0.
I20260808 08:25:00.481326 26077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:00.476041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.194:44277 (local address 127.22.235.193:58701)
0808 08:25:00.476292 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:00.476305 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:00.476462 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:00.476892 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:00.476895 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:00.476938 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:25:00.477069 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:00.477074 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.478061 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.478064 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:00.479467 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:00.479478 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.480445 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.480448 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:00.480553 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:00.480576 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 08:25:00.481115 (+ 539us) client_negotiation.cc:240] Negotiation successful
0808 08:25:00.481185 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":42,"spinlock_wait_cycles":4352,"thread_start_us":84,"threads_started":1}
I20260808 08:25:00.481637 25332 leader_election.cc:302] T 1addfb5834924e36bd36f6175c33c3c9 P 8a07e1d1baaf41689220531bd2692f11 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60a28860872c4a078b71afee9b53ace1, 8a07e1d1baaf41689220531bd2692f11; no voters:
I20260808 08:25:00.481727 25680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1addfb5834924e36bd36f6175c33c3c9" candidate_uuid: "8a07e1d1baaf41689220531bd2692f11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15380f70591f440a810a5e6e46fa89ad" is_pre_election: true
I20260808 08:25:00.481863 25680 raft_consensus.cc:2454] T 1addfb5834924e36bd36f6175c33c3c9 P 15380f70591f440a810a5e6e46fa89ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a07e1d1baaf41689220531bd2692f11 in term 0.
I20260808 08:25:00.481887 26040 raft_consensus.cc:2781] T 1addfb5834924e36bd36f6175c33c3c9 P 8a07e1d1baaf41689220531bd2692f11 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 08:25:00.481942 26040 raft_consensus.cc:482] T 1addfb5834924e36bd36f6175c33c3c9 P 8a07e1d1baaf41689220531bd2692f11 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 08:25:00.481987 26040 raft_consensus.cc:3037] T 1addfb5834924e36bd36f6175c33c3c9 P 8a07e1d1baaf41689220531bd2692f11 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:00.482909 26040 raft_consensus.cc:504] T 1addfb5834924e36bd36f6175c33c3c9 P 8a07e1d1baaf41689220531bd2692f11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a07e1d1baaf41689220531bd2692f11" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 42107 } } peers { permanent_uuid: "15380f70591f440a810a5e6e46fa89ad" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44277 } } peers { permanent_uuid: "60a28860872c4a078b71afee9b53ace1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39991 } }
I20260808 08:25:00.483227 26040 leader_election.cc:288] T 1addfb5834924e36bd36f6175c33c3c9 P 8a07e1d1baaf41689220531bd2692f11 [CANDIDATE]: Term 1 election: Requested vote from peers 15380f70591f440a810a5e6e46fa89ad (127.22.235.194:44277), 60a28860872c4a078b71afee9b53ace1 (127.22.235.195:39991)
I20260808 08:25:00.483530 25921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1addfb5834924e36bd36f6175c33c3c9" candidate_uuid: "8a07e1d1baaf41689220531bd2692f11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60a28860872c4a078b71afee9b53ace1"
I20260808 08:25:00.483534 25680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1addfb5834924e36bd36f6175c33c3c9" candidate_uuid: "8a07e1d1baaf41689220531bd2692f11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15380f70591f440a810a5e6e46fa89ad"
I20260808 08:25:00.483644 25921 raft_consensus.cc:3037] T 1addfb5834924e36bd36f6175c33c3c9 P 60a28860872c4a078b71afee9b53ace1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:00.483644 25680 raft_consensus.cc:3037] T 1addfb5834924e36bd36f6175c33c3c9 P 15380f70591f440a810a5e6e46fa89ad [term 0 FOLLOWER]: Advancing to term 1
W20260808 08:25:00.484423 25519 tablet.cc:2369] T 1addfb5834924e36bd36f6175c33c3c9 P 8a07e1d1baaf41689220531bd2692f11: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 08:25:00.484768 25921 raft_consensus.cc:2454] T 1addfb5834924e36bd36f6175c33c3c9 P 60a28860872c4a078b71afee9b53ace1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a07e1d1baaf41689220531bd2692f11 in term 1.
I20260808 08:25:00.484768 25680 raft_consensus.cc:2454] T 1addfb5834924e36bd36f6175c33c3c9 P 15380f70591f440a810a5e6e46fa89ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a07e1d1baaf41689220531bd2692f11 in term 1.
I20260808 08:25:00.485023 25332 leader_election.cc:302] T 1addfb5834924e36bd36f6175c33c3c9 P 8a07e1d1baaf41689220531bd2692f11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60a28860872c4a078b71afee9b53ace1, 8a07e1d1baaf41689220531bd2692f11; no voters:
I20260808 08:25:00.485184 26040 raft_consensus.cc:2781] T 1addfb5834924e36bd36f6175c33c3c9 P 8a07e1d1baaf41689220531bd2692f11 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:25:00.485388 26040 raft_consensus.cc:686] T 1addfb5834924e36bd36f6175c33c3c9 P 8a07e1d1baaf41689220531bd2692f11 [term 1 LEADER]: Becoming Leader. State: Replica: 8a07e1d1baaf41689220531bd2692f11, State: Running, Role: LEADER
I20260808 08:25:00.485626 26040 consensus_queue.cc:237] T 1addfb5834924e36bd36f6175c33c3c9 P 8a07e1d1baaf41689220531bd2692f11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a07e1d1baaf41689220531bd2692f11" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 42107 } } peers { permanent_uuid: "15380f70591f440a810a5e6e46fa89ad" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44277 } } peers { permanent_uuid: "60a28860872c4a078b71afee9b53ace1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39991 } }
I20260808 08:25:00.487840 25185 catalog_manager.cc:5495] T 1addfb5834924e36bd36f6175c33c3c9 P 8a07e1d1baaf41689220531bd2692f11 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a07e1d1baaf41689220531bd2692f11 (127.22.235.193). New cstate: current_term: 1 leader_uuid: "8a07e1d1baaf41689220531bd2692f11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a07e1d1baaf41689220531bd2692f11" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 42107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15380f70591f440a810a5e6e46fa89ad" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "60a28860872c4a078b71afee9b53ace1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39991 } health_report { overall_health: UNKNOWN } } }
Aug 08 08:25:00 dist-test-slave-f7th krb5kdc[25125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177499, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:25:00.522614 25273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:00.514349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33426 (local address 127.22.235.254:46093)
0808 08:25:00.514478 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:00.514483 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:00.514624 (+ 141us) server_negotiation.cc:407] Connection header received
0808 08:25:00.514737 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:00.514740 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:00.514788 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:00.514876 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:00.515472 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.516294 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.517137 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.517474 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.517575 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:00.519769 (+ 2194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:00.519772 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:00.519775 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:00.519795 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:00.521024 (+ 1229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:00.521536 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:00.521538 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:00.521539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:00.521605 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:00.521848 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:00.521850 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:00.521851 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:00.522012 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:00.522080 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:00.522377 (+ 297us) server_negotiation.cc:299] Negotiation successful
0808 08:25:00.522491 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":42}
I20260808 08:25:00.522769 26091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:00.514269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46093 (local address 127.0.0.1:33426)
0808 08:25:00.514540 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:00.514556 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:00.514657 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:00.514891 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:00.514893 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:00.515281 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:00.515373 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:00.515379 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.516408 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.516411 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:00.517037 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:00.517047 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.517570 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.517572 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:00.517633 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:00.518194 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:00.518205 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:00.519640 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:00.521126 (+ 1486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:00.521136 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:00.521143 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:00.521449 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:00.521679 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:00.521683 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:00.521685 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:00.521778 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:00.522097 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:00.522103 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:00.522228 (+ 125us) client_negotiation.cc:769] Sending connection context
0808 08:25:00.522344 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 08:25:00.522568 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":61,"thread_start_us":91,"threads_started":1}
I20260808 08:25:00.526861 25185 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:33426:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 08:25:00.527733 25185 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 08:25:00.532738 25412 tablet_service.cc:1467] Processing CreateTablet for tablet 0eed87514a614a959a17141ce26a56ef (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4c91bef7e795455095dfc11ae25e4fa0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 08:25:00.533178 25412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0eed87514a614a959a17141ce26a56ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:00.532728 25651 tablet_service.cc:1467] Processing CreateTablet for tablet 0eed87514a614a959a17141ce26a56ef (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4c91bef7e795455095dfc11ae25e4fa0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 08:25:00.532738 25883 tablet_service.cc:1467] Processing CreateTablet for tablet 0eed87514a614a959a17141ce26a56ef (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4c91bef7e795455095dfc11ae25e4fa0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 08:25:00.534268 25883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0eed87514a614a959a17141ce26a56ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:00.534265 25651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0eed87514a614a959a17141ce26a56ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:00.536021 26031 tablet_bootstrap.cc:492] T 0eed87514a614a959a17141ce26a56ef P 8a07e1d1baaf41689220531bd2692f11: Bootstrap starting.
I20260808 08:25:00.537019 26031 tablet_bootstrap.cc:654] T 0eed87514a614a959a17141ce26a56ef P 8a07e1d1baaf41689220531bd2692f11: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:00.537804 26033 tablet_bootstrap.cc:492] T 0eed87514a614a959a17141ce26a56ef P 15380f70591f440a810a5e6e46fa89ad: Bootstrap starting.
I20260808 08:25:00.538164 26032 tablet_bootstrap.cc:492] T 0eed87514a614a959a17141ce26a56ef P 60a28860872c4a078b71afee9b53ace1: Bootstrap starting.
I20260808 08:25:00.538715 26033 tablet_bootstrap.cc:654] T 0eed87514a614a959a17141ce26a56ef P 15380f70591f440a810a5e6e46fa89ad: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:00.538995 26032 tablet_bootstrap.cc:654] T 0eed87514a614a959a17141ce26a56ef P 60a28860872c4a078b71afee9b53ace1: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:00.539839 26033 tablet_bootstrap.cc:492] T 0eed87514a614a959a17141ce26a56ef P 15380f70591f440a810a5e6e46fa89ad: No bootstrap required, opened a new log
I20260808 08:25:00.539928 26033 ts_tablet_manager.cc:1397] T 0eed87514a614a959a17141ce26a56ef P 15380f70591f440a810a5e6e46fa89ad: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 08:25:00.540089 26031 tablet_bootstrap.cc:492] T 0eed87514a614a959a17141ce26a56ef P 8a07e1d1baaf41689220531bd2692f11: No bootstrap required, opened a new log
I20260808 08:25:00.540171 26031 ts_tablet_manager.cc:1397] T 0eed87514a614a959a17141ce26a56ef P 8a07e1d1baaf41689220531bd2692f11: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 08:25:00.540385 26033 raft_consensus.cc:348] T 0eed87514a614a959a17141ce26a56ef P 15380f70591f440a810a5e6e46fa89ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a07e1d1baaf41689220531bd2692f11" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 42107 } } peers { permanent_uuid: "15380f70591f440a810a5e6e46fa89ad" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44277 } } peers { permanent_uuid: "60a28860872c4a078b71afee9b53ace1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39991 } }
I20260808 08:25:00.540506 26033 raft_consensus.cc:374] T 0eed87514a614a959a17141ce26a56ef P 15380f70591f440a810a5e6e46fa89ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:00.540547 26033 raft_consensus.cc:729] T 0eed87514a614a959a17141ce26a56ef P 15380f70591f440a810a5e6e46fa89ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15380f70591f440a810a5e6e46fa89ad, State: Initialized, Role: FOLLOWER
I20260808 08:25:00.540575 26031 raft_consensus.cc:348] T 0eed87514a614a959a17141ce26a56ef P 8a07e1d1baaf41689220531bd2692f11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a07e1d1baaf41689220531bd2692f11" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 42107 } } peers { permanent_uuid: "15380f70591f440a810a5e6e46fa89ad" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44277 } } peers { permanent_uuid: "60a28860872c4a078b71afee9b53ace1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39991 } }
I20260808 08:25:00.540696 26031 raft_consensus.cc:374] T 0eed87514a614a959a17141ce26a56ef P 8a07e1d1baaf41689220531bd2692f11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:00.540729 26031 raft_consensus.cc:729] T 0eed87514a614a959a17141ce26a56ef P 8a07e1d1baaf41689220531bd2692f11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a07e1d1baaf41689220531bd2692f11, State: Initialized, Role: FOLLOWER
I20260808 08:25:00.540730 26033 consensus_queue.cc:260] T 0eed87514a614a959a17141ce26a56ef P 15380f70591f440a810a5e6e46fa89ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a07e1d1baaf41689220531bd2692f11" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 42107 } } peers { permanent_uuid: "15380f70591f440a810a5e6e46fa89ad" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44277 } } peers { permanent_uuid: "60a28860872c4a078b71afee9b53ace1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39991 } }
I20260808 08:25:00.540838 26031 consensus_queue.cc:260] T 0eed87514a614a959a17141ce26a56ef P 8a07e1d1baaf41689220531bd2692f11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a07e1d1baaf41689220531bd2692f11" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 42107 } } peers { permanent_uuid: "15380f70591f440a810a5e6e46fa89ad" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44277 } } peers { permanent_uuid: "60a28860872c4a078b71afee9b53ace1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39991 } }
I20260808 08:25:00.541003 26033 ts_tablet_manager.cc:1428] T 0eed87514a614a959a17141ce26a56ef P 15380f70591f440a810a5e6e46fa89ad: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 08:25:00.541069 26031 ts_tablet_manager.cc:1428] T 0eed87514a614a959a17141ce26a56ef P 8a07e1d1baaf41689220531bd2692f11: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 08:25:00.541225 26040 tablet_replica.cc:439] T 0eed87514a614a959a17141ce26a56ef P 8a07e1d1baaf41689220531bd2692f11: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a07e1d1baaf41689220531bd2692f11" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 42107 } } peers { permanent_uuid: "15380f70591f440a810a5e6e46fa89ad" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44277 } } peers { permanent_uuid: "60a28860872c4a078b71afee9b53ace1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39991 } } }
I20260808 08:25:00.541345 26040 tablet_replica.cc:442] T 0eed87514a614a959a17141ce26a56ef P 8a07e1d1baaf41689220531bd2692f11: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 08:25:00.541003 26041 tablet_replica.cc:439] T 0eed87514a614a959a17141ce26a56ef P 15380f70591f440a810a5e6e46fa89ad: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a07e1d1baaf41689220531bd2692f11" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 42107 } } peers { permanent_uuid: "15380f70591f440a810a5e6e46fa89ad" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44277 } } peers { permanent_uuid: "60a28860872c4a078b71afee9b53ace1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39991 } } }
I20260808 08:25:00.541777 26041 tablet_replica.cc:442] T 0eed87514a614a959a17141ce26a56ef P 15380f70591f440a810a5e6e46fa89ad: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 08:25:00.542697 26032 tablet_bootstrap.cc:492] T 0eed87514a614a959a17141ce26a56ef P 60a28860872c4a078b71afee9b53ace1: No bootstrap required, opened a new log
I20260808 08:25:00.542786 26032 ts_tablet_manager.cc:1397] T 0eed87514a614a959a17141ce26a56ef P 60a28860872c4a078b71afee9b53ace1: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20260808 08:25:00.543211 26032 raft_consensus.cc:348] T 0eed87514a614a959a17141ce26a56ef P 60a28860872c4a078b71afee9b53ace1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a07e1d1baaf41689220531bd2692f11" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 42107 } } peers { permanent_uuid: "15380f70591f440a810a5e6e46fa89ad" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44277 } } peers { permanent_uuid: "60a28860872c4a078b71afee9b53ace1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39991 } }
I20260808 08:25:00.543335 26032 raft_consensus.cc:374] T 0eed87514a614a959a17141ce26a56ef P 60a28860872c4a078b71afee9b53ace1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:00.543367 26032 raft_consensus.cc:729] T 0eed87514a614a959a17141ce26a56ef P 60a28860872c4a078b71afee9b53ace1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60a28860872c4a078b71afee9b53ace1, State: Initialized, Role: FOLLOWER
I20260808 08:25:00.543483 26032 consensus_queue.cc:260] T 0eed87514a614a959a17141ce26a56ef P 60a28860872c4a078b71afee9b53ace1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a07e1d1baaf41689220531bd2692f11" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 42107 } } peers { permanent_uuid: "15380f70591f440a810a5e6e46fa89ad" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44277 } } peers { permanent_uuid: "60a28860872c4a078b71afee9b53ace1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39991 } }
I20260808 08:25:00.543732 26032 ts_tablet_manager.cc:1428] T 0eed87514a614a959a17141ce26a56ef P 60a28860872c4a078b71afee9b53ace1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 08:25:00.543952 26038 tablet_replica.cc:439] T 0eed87514a614a959a17141ce26a56ef P 60a28860872c4a078b71afee9b53ace1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a07e1d1baaf41689220531bd2692f11" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 42107 } } peers { permanent_uuid: "15380f70591f440a810a5e6e46fa89ad" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44277 } } peers { permanent_uuid: "60a28860872c4a078b71afee9b53ace1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39991 } } }
I20260808 08:25:00.544086 26038 tablet_replica.cc:442] T 0eed87514a614a959a17141ce26a56ef P 60a28860872c4a078b71afee9b53ace1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 08:25:00.586211 26038 raft_consensus.cc:482] T 0eed87514a614a959a17141ce26a56ef P 60a28860872c4a078b71afee9b53ace1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 08:25:00.586349 26038 raft_consensus.cc:504] T 0eed87514a614a959a17141ce26a56ef P 60a28860872c4a078b71afee9b53ace1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a07e1d1baaf41689220531bd2692f11" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 42107 } } peers { permanent_uuid: "15380f70591f440a810a5e6e46fa89ad" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44277 } } peers { permanent_uuid: "60a28860872c4a078b71afee9b53ace1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39991 } }
I20260808 08:25:00.586746 26038 leader_election.cc:288] T 0eed87514a614a959a17141ce26a56ef P 60a28860872c4a078b71afee9b53ace1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a07e1d1baaf41689220531bd2692f11 (127.22.235.193:42107), 15380f70591f440a810a5e6e46fa89ad (127.22.235.194:44277)
I20260808 08:25:00.587205 25680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eed87514a614a959a17141ce26a56ef" candidate_uuid: "60a28860872c4a078b71afee9b53ace1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15380f70591f440a810a5e6e46fa89ad" is_pre_election: true
I20260808 08:25:00.587191 25440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eed87514a614a959a17141ce26a56ef" candidate_uuid: "60a28860872c4a078b71afee9b53ace1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a07e1d1baaf41689220531bd2692f11" is_pre_election: true
I20260808 08:25:00.587345 25440 raft_consensus.cc:2454] T 0eed87514a614a959a17141ce26a56ef P 8a07e1d1baaf41689220531bd2692f11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60a28860872c4a078b71afee9b53ace1 in term 0.
I20260808 08:25:00.587347 25680 raft_consensus.cc:2454] T 0eed87514a614a959a17141ce26a56ef P 15380f70591f440a810a5e6e46fa89ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60a28860872c4a078b71afee9b53ace1 in term 0.
I20260808 08:25:00.587688 25801 leader_election.cc:302] T 0eed87514a614a959a17141ce26a56ef P 60a28860872c4a078b71afee9b53ace1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 15380f70591f440a810a5e6e46fa89ad, 60a28860872c4a078b71afee9b53ace1, 8a07e1d1baaf41689220531bd2692f11; no voters:
I20260808 08:25:00.587863 26038 raft_consensus.cc:2781] T 0eed87514a614a959a17141ce26a56ef P 60a28860872c4a078b71afee9b53ace1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 08:25:00.587916 26038 raft_consensus.cc:482] T 0eed87514a614a959a17141ce26a56ef P 60a28860872c4a078b71afee9b53ace1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 08:25:00.587960 26038 raft_consensus.cc:3037] T 0eed87514a614a959a17141ce26a56ef P 60a28860872c4a078b71afee9b53ace1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:00.589130 26038 raft_consensus.cc:504] T 0eed87514a614a959a17141ce26a56ef P 60a28860872c4a078b71afee9b53ace1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a07e1d1baaf41689220531bd2692f11" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 42107 } } peers { permanent_uuid: "15380f70591f440a810a5e6e46fa89ad" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44277 } } peers { permanent_uuid: "60a28860872c4a078b71afee9b53ace1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39991 } }
I20260808 08:25:00.589454 26038 leader_election.cc:288] T 0eed87514a614a959a17141ce26a56ef P 60a28860872c4a078b71afee9b53ace1 [CANDIDATE]: Term 1 election: Requested vote from peers 8a07e1d1baaf41689220531bd2692f11 (127.22.235.193:42107), 15380f70591f440a810a5e6e46fa89ad (127.22.235.194:44277)
I20260808 08:25:00.589762 25680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eed87514a614a959a17141ce26a56ef" candidate_uuid: "60a28860872c4a078b71afee9b53ace1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15380f70591f440a810a5e6e46fa89ad"
I20260808 08:25:00.589793 25440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0eed87514a614a959a17141ce26a56ef" candidate_uuid: "60a28860872c4a078b71afee9b53ace1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a07e1d1baaf41689220531bd2692f11"
I20260808 08:25:00.589864 25680 raft_consensus.cc:3037] T 0eed87514a614a959a17141ce26a56ef P 15380f70591f440a810a5e6e46fa89ad [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:00.589916 25440 raft_consensus.cc:3037] T 0eed87514a614a959a17141ce26a56ef P 8a07e1d1baaf41689220531bd2692f11 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:00.590806 25680 raft_consensus.cc:2454] T 0eed87514a614a959a17141ce26a56ef P 15380f70591f440a810a5e6e46fa89ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60a28860872c4a078b71afee9b53ace1 in term 1.
I20260808 08:25:00.591081 25802 leader_election.cc:302] T 0eed87514a614a959a17141ce26a56ef P 60a28860872c4a078b71afee9b53ace1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15380f70591f440a810a5e6e46fa89ad, 60a28860872c4a078b71afee9b53ace1; no voters:
I20260808 08:25:00.591132 25440 raft_consensus.cc:2454] T 0eed87514a614a959a17141ce26a56ef P 8a07e1d1baaf41689220531bd2692f11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60a28860872c4a078b71afee9b53ace1 in term 1.
I20260808 08:25:00.591272 26038 raft_consensus.cc:2781] T 0eed87514a614a959a17141ce26a56ef P 60a28860872c4a078b71afee9b53ace1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:25:00.591363 26038 raft_consensus.cc:686] T 0eed87514a614a959a17141ce26a56ef P 60a28860872c4a078b71afee9b53ace1 [term 1 LEADER]: Becoming Leader. State: Replica: 60a28860872c4a078b71afee9b53ace1, State: Running, Role: LEADER
I20260808 08:25:00.591511 26038 consensus_queue.cc:237] T 0eed87514a614a959a17141ce26a56ef P 60a28860872c4a078b71afee9b53ace1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a07e1d1baaf41689220531bd2692f11" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 42107 } } peers { permanent_uuid: "15380f70591f440a810a5e6e46fa89ad" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44277 } } peers { permanent_uuid: "60a28860872c4a078b71afee9b53ace1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39991 } }
I20260808 08:25:00.592163 26038 tablet_replica.cc:439] T 0eed87514a614a959a17141ce26a56ef P 60a28860872c4a078b71afee9b53ace1: TxnStatusTablet state changed. Reason: New leader 60a28860872c4a078b71afee9b53ace1. Latest consensus state: current_term: 1 leader_uuid: "60a28860872c4a078b71afee9b53ace1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a07e1d1baaf41689220531bd2692f11" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 42107 } } peers { permanent_uuid: "15380f70591f440a810a5e6e46fa89ad" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44277 } } peers { permanent_uuid: "60a28860872c4a078b71afee9b53ace1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39991 } } }
I20260808 08:25:00.592263 26038 tablet_replica.cc:442] T 0eed87514a614a959a17141ce26a56ef P 60a28860872c4a078b71afee9b53ace1: This TxnStatusTablet replica's current role is: LEADER
I20260808 08:25:00.592630 26097 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 08:25:00.592844 26097 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 08:25:00.593379 25190 catalog_manager.cc:5495] T 0eed87514a614a959a17141ce26a56ef P 60a28860872c4a078b71afee9b53ace1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 60a28860872c4a078b71afee9b53ace1 (127.22.235.195). New cstate: current_term: 1 leader_uuid: "60a28860872c4a078b71afee9b53ace1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a07e1d1baaf41689220531bd2692f11" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 42107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15380f70591f440a810a5e6e46fa89ad" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "60a28860872c4a078b71afee9b53ace1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39991 } health_report { overall_health: HEALTHY } } }
I20260808 08:25:00.609509 26091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:00.606364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.195:39991 (local address 127.0.0.1:56604)
0808 08:25:00.606521 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:00.606536 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:00.606642 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:00.606905 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:00.606908 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:00.606915 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 08:25:00.607029 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:00.607034 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.607730 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.607734 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:00.608335 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:00.608343 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.608814 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.608817 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:00.608903 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:00.608947 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 08:25:00.609226 (+ 279us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 08:25:00.609241 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 08:25:00.609301 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 08:25:00.609387 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 08:25:00.609513 26029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:00.606436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56604 (local address 127.22.235.195:39991)
0808 08:25:00.606582 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:00.606586 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:00.606600 (+ 14us) server_negotiation.cc:407] Connection header received
0808 08:25:00.606751 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:00.606754 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:00.606815 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:00.606875 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:25:00.607124 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.607632 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.608443 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.608713 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.608802 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:00.609076 (+ 274us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:25:00.609150 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 08:25:00.609207 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:00.609325 (+ 118us) server_negotiation.cc:299] Negotiation successful
0808 08:25:00.609375 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 08:25:00.612098 25680 raft_consensus.cc:1261] T 0eed87514a614a959a17141ce26a56ef P 15380f70591f440a810a5e6e46fa89ad [term 1 FOLLOWER]: Refusing update from remote peer 60a28860872c4a078b71afee9b53ace1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:25:00.612118 25440 raft_consensus.cc:1261] T 0eed87514a614a959a17141ce26a56ef P 8a07e1d1baaf41689220531bd2692f11 [term 1 FOLLOWER]: Refusing update from remote peer 60a28860872c4a078b71afee9b53ace1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:25:00.612499 26038 consensus_queue.cc:1035] T 0eed87514a614a959a17141ce26a56ef P 60a28860872c4a078b71afee9b53ace1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15380f70591f440a810a5e6e46fa89ad" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:25:00.612676 26064 consensus_queue.cc:1035] T 0eed87514a614a959a17141ce26a56ef P 60a28860872c4a078b71afee9b53ace1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a07e1d1baaf41689220531bd2692f11" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 42107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:25:00.614108 26040 tablet_replica.cc:439] T 0eed87514a614a959a17141ce26a56ef P 8a07e1d1baaf41689220531bd2692f11: TxnStatusTablet state changed. Reason: New leader 60a28860872c4a078b71afee9b53ace1. Latest consensus state: current_term: 1 leader_uuid: "60a28860872c4a078b71afee9b53ace1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a07e1d1baaf41689220531bd2692f11" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 42107 } } peers { permanent_uuid: "15380f70591f440a810a5e6e46fa89ad" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44277 } } peers { permanent_uuid: "60a28860872c4a078b71afee9b53ace1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39991 } } }
I20260808 08:25:00.614217 26040 tablet_replica.cc:442] T 0eed87514a614a959a17141ce26a56ef P 8a07e1d1baaf41689220531bd2692f11: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 08:25:00.614904 26041 tablet_replica.cc:439] T 0eed87514a614a959a17141ce26a56ef P 15380f70591f440a810a5e6e46fa89ad: TxnStatusTablet state changed. Reason: New leader 60a28860872c4a078b71afee9b53ace1. Latest consensus state: current_term: 1 leader_uuid: "60a28860872c4a078b71afee9b53ace1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a07e1d1baaf41689220531bd2692f11" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 42107 } } peers { permanent_uuid: "15380f70591f440a810a5e6e46fa89ad" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44277 } } peers { permanent_uuid: "60a28860872c4a078b71afee9b53ace1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39991 } } }
I20260808 08:25:00.615041 26041 tablet_replica.cc:442] T 0eed87514a614a959a17141ce26a56ef P 15380f70591f440a810a5e6e46fa89ad: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 08:25:00.616367 26038 tablet_replica.cc:439] T 0eed87514a614a959a17141ce26a56ef P 60a28860872c4a078b71afee9b53ace1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "60a28860872c4a078b71afee9b53ace1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a07e1d1baaf41689220531bd2692f11" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 42107 } } peers { permanent_uuid: "15380f70591f440a810a5e6e46fa89ad" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44277 } } peers { permanent_uuid: "60a28860872c4a078b71afee9b53ace1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39991 } } }
I20260808 08:25:00.616473 26038 tablet_replica.cc:442] T 0eed87514a614a959a17141ce26a56ef P 60a28860872c4a078b71afee9b53ace1: This TxnStatusTablet replica's current role is: LEADER
I20260808 08:25:00.617560 26047 tablet_replica.cc:439] T 0eed87514a614a959a17141ce26a56ef P 60a28860872c4a078b71afee9b53ace1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "60a28860872c4a078b71afee9b53ace1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a07e1d1baaf41689220531bd2692f11" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 42107 } } peers { permanent_uuid: "15380f70591f440a810a5e6e46fa89ad" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44277 } } peers { permanent_uuid: "60a28860872c4a078b71afee9b53ace1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39991 } } }
I20260808 08:25:00.617662 26047 tablet_replica.cc:442] T 0eed87514a614a959a17141ce26a56ef P 60a28860872c4a078b71afee9b53ace1: This TxnStatusTablet replica's current role is: LEADER
I20260808 08:25:00.618276 26053 mvcc.cc:205] Tried to move back new op lower bound from 7316183042503114752 to 7316183042423971840. Current Snapshot: MvccSnapshot[applied={T|T < 7316183042503114752 or (T in {7316183042503114752})}]
I20260808 08:25:00.618621 26041 tablet_replica.cc:439] T 0eed87514a614a959a17141ce26a56ef P 15380f70591f440a810a5e6e46fa89ad: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "60a28860872c4a078b71afee9b53ace1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a07e1d1baaf41689220531bd2692f11" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 42107 } } peers { permanent_uuid: "15380f70591f440a810a5e6e46fa89ad" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44277 } } peers { permanent_uuid: "60a28860872c4a078b71afee9b53ace1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39991 } } }
I20260808 08:25:00.618716 26041 tablet_replica.cc:442] T 0eed87514a614a959a17141ce26a56ef P 15380f70591f440a810a5e6e46fa89ad: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 08:25:00.619673 26040 tablet_replica.cc:439] T 0eed87514a614a959a17141ce26a56ef P 8a07e1d1baaf41689220531bd2692f11: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "60a28860872c4a078b71afee9b53ace1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a07e1d1baaf41689220531bd2692f11" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 42107 } } peers { permanent_uuid: "15380f70591f440a810a5e6e46fa89ad" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44277 } } peers { permanent_uuid: "60a28860872c4a078b71afee9b53ace1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39991 } } }
I20260808 08:25:00.619779 26040 tablet_replica.cc:442] T 0eed87514a614a959a17141ce26a56ef P 8a07e1d1baaf41689220531bd2692f11: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 08:25:00.620846 26054 mvcc.cc:205] Tried to move back new op lower bound from 7316183042503114752 to 7316183042423971840. Current Snapshot: MvccSnapshot[applied={T|T < 7316183042503114752 or (T in {7316183042503114752})}]
I20260808 08:25:00.626577 26020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:00.622974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41316 (local address 127.22.235.193:42107)
0808 08:25:00.623078 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:00.623083 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:00.623135 (+ 52us) server_negotiation.cc:407] Connection header received
0808 08:25:00.623307 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:00.623311 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:00.623363 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:00.623447 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:25:00.623676 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.624213 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.625095 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.625368 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.625479 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:00.625798 (+ 319us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:25:00.626222 (+ 424us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 08:25:00.626321 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:00.626391 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 08:25:00.626442 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 08:25:00.636679 26029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:00.633604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:54157 (local address 127.22.235.195:39991)
0808 08:25:00.633706 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:00.633709 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:00.633938 (+ 229us) server_negotiation.cc:407] Connection header received
0808 08:25:00.634064 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:00.634068 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:00.634113 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:00.634172 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:25:00.634392 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.634868 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.635690 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.635895 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.635978 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:00.636246 (+ 268us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:25:00.636312 (+ 66us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 08:25:00.636368 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:00.636455 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 08:25:00.636535 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 08:25:00.636933 26108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:00.633536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.195:39991 (local address 127.22.235.193:54157)
0808 08:25:00.633857 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:00.633868 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:00.633965 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:00.634187 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:00.634190 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:00.634196 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 08:25:00.634300 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:00.634305 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.634964 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.634968 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:00.635584 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:00.635592 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.635987 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.635990 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:00.636083 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:00.636126 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 08:25:00.636383 (+ 257us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 08:25:00.636389 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 08:25:00.636446 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 08:25:00.636522 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":99,"threads_started":1}
I20260808 08:25:00.644011 25921 raft_consensus.cc:1261] T 1addfb5834924e36bd36f6175c33c3c9 P 60a28860872c4a078b71afee9b53ace1 [term 1 FOLLOWER]: Refusing update from remote peer 8a07e1d1baaf41689220531bd2692f11: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:25:00.644007 25680 raft_consensus.cc:1261] T 1addfb5834924e36bd36f6175c33c3c9 P 15380f70591f440a810a5e6e46fa89ad [term 1 FOLLOWER]: Refusing update from remote peer 8a07e1d1baaf41689220531bd2692f11: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:25:00.644469 26040 consensus_queue.cc:1035] T 1addfb5834924e36bd36f6175c33c3c9 P 8a07e1d1baaf41689220531bd2692f11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60a28860872c4a078b71afee9b53ace1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:25:00.644712 26079 consensus_queue.cc:1035] T 1addfb5834924e36bd36f6175c33c3c9 P 8a07e1d1baaf41689220531bd2692f11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15380f70591f440a810a5e6e46fa89ad" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:25:00.668203 26020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:00.664729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:38967 (local address 127.22.235.193:42107)
0808 08:25:00.665049 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:00.665054 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:00.665067 (+ 13us) server_negotiation.cc:407] Connection header received
0808 08:25:00.665109 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:00.665112 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:00.665160 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:00.665228 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:25:00.665539 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.666248 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.667169 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.667387 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.667471 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:00.667735 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:25:00.667834 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 08:25:00.667896 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:00.668023 (+ 127us) server_negotiation.cc:299] Negotiation successful
0808 08:25:00.668072 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243}
I20260808 08:25:00.668203 26002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:00.664469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:42107 (local address 127.22.235.195:38967)
0808 08:25:00.664785 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:00.664801 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:00.664885 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:00.665330 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:00.665332 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:00.665339 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 08:25:00.665441 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:00.665447 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.666347 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.666350 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:00.667066 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:00.667076 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.667478 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.667482 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:00.667573 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:00.667618 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 08:25:00.667935 (+ 317us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 08:25:00.667942 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 08:25:00.668007 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 08:25:00.668085 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187}
I20260808 08:25:00.694371 25190 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33410:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:25:00.694552 25190 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33410:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:25:00.696861 25190 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f3651710b85e4fe9be5fd28aff89bf89: Sending DeleteTablet for 3 replicas of tablet 1addfb5834924e36bd36f6175c33c3c9
I20260808 08:25:00.697804 25651 tablet_service.cc:1514] Processing DeleteTablet for tablet 1addfb5834924e36bd36f6175c33c3c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:25:00 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:33162
I20260808 08:25:00.698004 26067 tablet_replica.cc:330] T 1addfb5834924e36bd36f6175c33c3c9 P 15380f70591f440a810a5e6e46fa89ad: stopping tablet replica
I20260808 08:25:00.698133 26067 raft_consensus.cc:2229] T 1addfb5834924e36bd36f6175c33c3c9 P 15380f70591f440a810a5e6e46fa89ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 08:25:00.698136 25412 tablet_service.cc:1514] Processing DeleteTablet for tablet 1addfb5834924e36bd36f6175c33c3c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:25:00 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:41306
I20260808 08:25:00.698251 26067 raft_consensus.cc:2258] T 1addfb5834924e36bd36f6175c33c3c9 P 15380f70591f440a810a5e6e46fa89ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:25:00.698338 26068 tablet_replica.cc:330] T 1addfb5834924e36bd36f6175c33c3c9 P 8a07e1d1baaf41689220531bd2692f11: stopping tablet replica
I20260808 08:25:00.698431 26068 raft_consensus.cc:2229] T 1addfb5834924e36bd36f6175c33c3c9 P 8a07e1d1baaf41689220531bd2692f11 [term 1 LEADER]: Raft consensus shutting down.
I20260808 08:25:00.698709 26068 raft_consensus.cc:2258] T 1addfb5834924e36bd36f6175c33c3c9 P 8a07e1d1baaf41689220531bd2692f11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:25:00.699996 26067 ts_tablet_manager.cc:1905] T 1addfb5834924e36bd36f6175c33c3c9 P 15380f70591f440a810a5e6e46fa89ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:25:00.700744 25883 tablet_service.cc:1514] Processing DeleteTablet for tablet 1addfb5834924e36bd36f6175c33c3c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:25:00 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:56588
I20260808 08:25:00.700947 26071 tablet_replica.cc:330] T 1addfb5834924e36bd36f6175c33c3c9 P 60a28860872c4a078b71afee9b53ace1: stopping tablet replica
I20260808 08:25:00.701079 26071 raft_consensus.cc:2229] T 1addfb5834924e36bd36f6175c33c3c9 P 60a28860872c4a078b71afee9b53ace1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 08:25:00.701195 26071 raft_consensus.cc:2258] T 1addfb5834924e36bd36f6175c33c3c9 P 60a28860872c4a078b71afee9b53ace1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:25:00.701431 26068 ts_tablet_manager.cc:1905] T 1addfb5834924e36bd36f6175c33c3c9 P 8a07e1d1baaf41689220531bd2692f11: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:25:00.704164 26068 ts_tablet_manager.cc:1918] T 1addfb5834924e36bd36f6175c33c3c9 P 8a07e1d1baaf41689220531bd2692f11: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 08:25:00.704244 26068 log.cc:1196] T 1addfb5834924e36bd36f6175c33c3c9 P 8a07e1d1baaf41689220531bd2692f11: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-0/wal/wals/1addfb5834924e36bd36f6175c33c3c9
I20260808 08:25:00.704476 26068 ts_tablet_manager.cc:1939] T 1addfb5834924e36bd36f6175c33c3c9 P 8a07e1d1baaf41689220531bd2692f11: Deleting consensus metadata
I20260808 08:25:00.704986 26071 ts_tablet_manager.cc:1905] T 1addfb5834924e36bd36f6175c33c3c9 P 60a28860872c4a078b71afee9b53ace1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:25:00.705168 25164 catalog_manager.cc:4841] TS 8a07e1d1baaf41689220531bd2692f11 (127.22.235.193:42107): tablet 1addfb5834924e36bd36f6175c33c3c9 (table test-table [id=a3249cf243674e13be890b250d2856ec]) successfully deleted
I20260808 08:25:00.707527 26071 ts_tablet_manager.cc:1918] T 1addfb5834924e36bd36f6175c33c3c9 P 60a28860872c4a078b71afee9b53ace1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 08:25:00.707608 26071 log.cc:1196] T 1addfb5834924e36bd36f6175c33c3c9 P 60a28860872c4a078b71afee9b53ace1: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-2/wal/wals/1addfb5834924e36bd36f6175c33c3c9
I20260808 08:25:00.707847 26071 ts_tablet_manager.cc:1939] T 1addfb5834924e36bd36f6175c33c3c9 P 60a28860872c4a078b71afee9b53ace1: Deleting consensus metadata
I20260808 08:25:00.708493 25164 catalog_manager.cc:4841] TS 60a28860872c4a078b71afee9b53ace1 (127.22.235.195:39991): tablet 1addfb5834924e36bd36f6175c33c3c9 (table test-table [id=a3249cf243674e13be890b250d2856ec]) successfully deleted
I20260808 08:25:00.708720 26067 ts_tablet_manager.cc:1918] T 1addfb5834924e36bd36f6175c33c3c9 P 15380f70591f440a810a5e6e46fa89ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 08:25:00.708850 26067 log.cc:1196] T 1addfb5834924e36bd36f6175c33c3c9 P 15380f70591f440a810a5e6e46fa89ad: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786177493204855-23471-0/minicluster-data/ts-1/wal/wals/1addfb5834924e36bd36f6175c33c3c9
I20260808 08:25:00.709156 26067 ts_tablet_manager.cc:1939] T 1addfb5834924e36bd36f6175c33c3c9 P 15380f70591f440a810a5e6e46fa89ad: Deleting consensus metadata
I20260808 08:25:00.709849 25165 catalog_manager.cc:4841] TS 15380f70591f440a810a5e6e46fa89ad (127.22.235.194:44277): tablet 1addfb5834924e36bd36f6175c33c3c9 (table test-table [id=a3249cf243674e13be890b250d2856ec]) successfully deleted
Aug 08 08:25:00 dist-test-slave-f7th krb5kdc[25125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177500, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.235.193@KRBTEST.COM
I20260808 08:25:00.720791 26020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:00.699082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48504 (local address 127.22.235.193:42107)
0808 08:25:00.699196 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:00.699200 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:00.709019 (+ 9819us) server_negotiation.cc:407] Connection header received
0808 08:25:00.709212 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:00.709216 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:00.709262 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:00.709309 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:00.710243 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.710980 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.712169 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.712490 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.712595 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:00.716317 (+ 3722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:00.716330 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:00.716373 (+ 43us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:00.716402 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:00.718619 (+ 2217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:00.719129 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:00.719136 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:00.719138 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:00.719240 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:00.719533 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:00.719537 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:00.719538 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:00.719840 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:00.720107 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:00.720310 (+ 203us) server_negotiation.cc:299] Negotiation successful
0808 08:25:00.720432 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260808 08:25:00.721338 25509 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48504
I20260808 08:25:00.732820 25273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:00.723248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38632 (local address 127.22.235.254:46093)
0808 08:25:00.723366 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:00.723371 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:00.723870 (+ 499us) server_negotiation.cc:407] Connection header received
0808 08:25:00.724060 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:00.724064 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:00.724116 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:00.724167 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:00.725112 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.725932 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.727123 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:00.727463 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:00.727535 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:00.729258 (+ 1723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:00.729262 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:00.729265 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:00.729291 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:00.730899 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:00.731364 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:00.731368 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:00.731370 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:00.731450 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:00.731739 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:00.731743 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:00.731745 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:00.731969 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:00.732081 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:00.732490 (+ 409us) server_negotiation.cc:299] Negotiation successful
0808 08:25:00.732607 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260808 08:25:00.733783 25190 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38632
I20260808 08:25:00.738327 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 25307
I20260808 08:25:00.756947 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 25536
I20260808 08:25:00.792313 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 25783
I20260808 08:25:00.812161 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 25144
2026-08-08T08:25:00Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2832 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 08:25:00 dist-test-slave-f7th krb5kdc[26123](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 08:25:00 dist-test-slave-f7th krb5kdc[26123](info): set up 2 sockets
Aug 08 08:25:00 dist-test-slave-f7th krb5kdc[26123](info): commencing operation
krb5kdc: starting...
W20260808 08:25:01.975884 23471 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Aug 08 08:25:02 dist-test-slave-f7th krb5kdc[26123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T08:25:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:25:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.235.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
I20260808 08:25:02.097023 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:36447
--webserver_interface=127.22.235.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:44915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:36447
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:25:02.208330 26143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:02.208601 26143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:02.208693 26143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:02.212690 26143 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:25:02.212749 26143 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 08:25:02.212764 26143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:02.212785 26143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:02.212821 26143 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:25:02.212848 26143 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:25:02.218132 26143 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:44915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:36447
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:36447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.235.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26143
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:02.219159 26143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:02.220180 26143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:02.228632 26152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:02.228645 26153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:02.228674 26155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:02.228825 26143 server_base.cc:1034] running on GCE node
I20260808 08:25:02.229486 26143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:02.230651 26143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:02.231869 26143 hybrid_clock.cc:648] HybridClock initialized: now 1786177502231853 us; error 30 us; skew 500 ppm
Aug 08 08:25:02 dist-test-slave-f7th krb5kdc[26123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177502, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:02.234503 26143 init.cc:377] Logged in from keytab as kudu/127.22.235.254@KRBTEST.COM (short username kudu)
I20260808 08:25:02.235435 26143 webserver.cc:460] Webserver started at http://127.22.235.254:35767/ using document root <none> and password file <none>
I20260808 08:25:02.235983 26143 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:02.236049 26143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:02.236256 26143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:02.238291 26143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "14fef20eb9444ae995182431d0545751"
format_stamp: "Formatted at 2026-08-08 08:25:02 on dist-test-slave-f7th"
server_key: "64562bcb8c2584b13b33f1852515c475"
server_key_iv: "5a0365ba4b2f0c3818943d2901918527"
server_key_version: "encryptionkey@0"
I20260808 08:25:02.238849 26143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "14fef20eb9444ae995182431d0545751"
format_stamp: "Formatted at 2026-08-08 08:25:02 on dist-test-slave-f7th"
server_key: "64562bcb8c2584b13b33f1852515c475"
server_key_iv: "5a0365ba4b2f0c3818943d2901918527"
server_key_version: "encryptionkey@0"
I20260808 08:25:02.242504 26143 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260808 08:25:02.244987 26167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:02.246188 26143 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:25:02.246301 26143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "14fef20eb9444ae995182431d0545751"
format_stamp: "Formatted at 2026-08-08 08:25:02 on dist-test-slave-f7th"
server_key: "64562bcb8c2584b13b33f1852515c475"
server_key_iv: "5a0365ba4b2f0c3818943d2901918527"
server_key_version: "encryptionkey@0"
I20260808 08:25:02.246392 26143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:02.274081 26143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:02.276841 26143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:02.277024 26143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:02.284591 26143 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.254:36447
I20260808 08:25:02.284631 26273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.254:36447 every 8 connection(s)
I20260808 08:25:02.285524 26143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:25:02.288419 26276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:02.293326 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 26143
I20260808 08:25:02.293432 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:25:02.293659 23471 external_mini_cluster.cc:1428] Setting key 4e7c01e1a60fae9b1119dbaf0f3fee5f
I20260808 08:25:02.294059 26276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14fef20eb9444ae995182431d0545751: Bootstrap starting.
I20260808 08:25:02.296514 26276 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14fef20eb9444ae995182431d0545751: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:02.297271 26276 log.cc:824] T 00000000000000000000000000000000 P 14fef20eb9444ae995182431d0545751: Log is configured to *not* fsync() on all Append() calls
I20260808 08:25:02.299086 26276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14fef20eb9444ae995182431d0545751: No bootstrap required, opened a new log
Aug 08 08:25:02 dist-test-slave-f7th krb5kdc[26123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:25:02.302759 26276 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14fef20eb9444ae995182431d0545751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14fef20eb9444ae995182431d0545751" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 36447 } }
I20260808 08:25:02.303049 26276 raft_consensus.cc:374] T 00000000000000000000000000000000 P 14fef20eb9444ae995182431d0545751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:02.303114 26276 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14fef20eb9444ae995182431d0545751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14fef20eb9444ae995182431d0545751, State: Initialized, Role: FOLLOWER
I20260808 08:25:02.303594 26276 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14fef20eb9444ae995182431d0545751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14fef20eb9444ae995182431d0545751" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 36447 } }
I20260808 08:25:02.303741 26276 raft_consensus.cc:388] T 00000000000000000000000000000000 P 14fef20eb9444ae995182431d0545751 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:25:02.303817 26276 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14fef20eb9444ae995182431d0545751 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:25:02.303920 26276 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14fef20eb9444ae995182431d0545751 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:02.304776 26276 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14fef20eb9444ae995182431d0545751 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14fef20eb9444ae995182431d0545751" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 36447 } }
I20260808 08:25:02.305101 26276 leader_election.cc:302] T 00000000000000000000000000000000 P 14fef20eb9444ae995182431d0545751 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 14fef20eb9444ae995182431d0545751; no voters:
I20260808 08:25:02.305373 26276 leader_election.cc:288] T 00000000000000000000000000000000 P 14fef20eb9444ae995182431d0545751 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:25:02.305466 26288 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 14fef20eb9444ae995182431d0545751 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:25:02.305698 26288 raft_consensus.cc:686] T 00000000000000000000000000000000 P 14fef20eb9444ae995182431d0545751 [term 1 LEADER]: Becoming Leader. State: Replica: 14fef20eb9444ae995182431d0545751, State: Running, Role: LEADER
I20260808 08:25:02.306011 26288 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14fef20eb9444ae995182431d0545751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14fef20eb9444ae995182431d0545751" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 36447 } }
I20260808 08:25:02.306587 26276 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14fef20eb9444ae995182431d0545751 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:25:02.307228 26288 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14fef20eb9444ae995182431d0545751 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14fef20eb9444ae995182431d0545751. Latest consensus state: current_term: 1 leader_uuid: "14fef20eb9444ae995182431d0545751" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14fef20eb9444ae995182431d0545751" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 36447 } } }
I20260808 08:25:02.307346 26288 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14fef20eb9444ae995182431d0545751 [sys.catalog]: This master's current role is: LEADER
I20260808 08:25:02.307677 26289 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14fef20eb9444ae995182431d0545751 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "14fef20eb9444ae995182431d0545751" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14fef20eb9444ae995182431d0545751" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 36447 } } }
I20260808 08:25:02.307772 26289 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14fef20eb9444ae995182431d0545751 [sys.catalog]: This master's current role is: LEADER
I20260808 08:25:02.307830 26284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:02.295023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33226 (local address 127.22.235.254:36447)
0808 08:25:02.295485 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:02.295508 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:02.295544 (+ 36us) server_negotiation.cc:407] Connection header received
0808 08:25:02.296154 (+ 610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:02.296171 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:02.296485 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:02.296849 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:02.297632 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:02.298599 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:02.299475 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:02.299940 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:02.300083 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:02.302645 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:02.302651 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:02.302664 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:02.302691 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:02.304343 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:02.304783 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:02.304789 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:02.304795 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:02.304932 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:02.305187 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:02.305190 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:02.305192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:02.305697 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:02.306465 (+ 768us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:02.306612 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 08:25:02.306910 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":148,"threads_started":1}
I20260808 08:25:02.308439 26296 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:25:02.311450 26296 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:25:02.317054 26296 catalog_manager.cc:1347] Generated new cluster ID: a268df49a9594e99af7de4ede2277637
I20260808 08:25:02.317117 26296 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:25:02.327960 26296 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:25:02.328930 26296 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:25:02.345875 26296 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 14fef20eb9444ae995182431d0545751: Generated new TSK 0
I20260808 08:25:02.346596 26296 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.235.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab.
Entry for principal kudu/127.22.235.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab.
I20260808 08:25:02.381915 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.193:0
--local_ip_for_outbound_sockets=127.22.235.193
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:36447
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab
--principal=kudu/127.22.235.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:44915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:25:02.500020 26318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:02.500296 26318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:02.500365 26318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:02.504384 26318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:02.504449 26318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:02.504505 26318 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 08:25:02.504532 26318 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 08:25:02.504581 26318 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.193
I20260808 08:25:02.509965 26318 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:44915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab
--principal=kudu/127.22.235.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.235.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.235.254:36447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26318
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.193
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:02.511013 26318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:02.512024 26318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:02.521366 26327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:02.521375 26325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:02.521401 26324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:02.521721 26318 server_base.cc:1034] running on GCE node
I20260808 08:25:02.522112 26318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:02.522735 26318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:02.523908 26318 hybrid_clock.cc:648] HybridClock initialized: now 1786177502523877 us; error 41 us; skew 500 ppm
Aug 08 08:25:02 dist-test-slave-f7th krb5kdc[26123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177502, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:02.526633 26318 init.cc:377] Logged in from keytab as kudu/127.22.235.193@KRBTEST.COM (short username kudu)
I20260808 08:25:02.527530 26318 webserver.cc:460] Webserver started at http://127.22.235.193:35925/ using document root <none> and password file <none>
I20260808 08:25:02.528054 26318 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:02.528110 26318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:02.528331 26318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:02.530360 26318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-0/data/instance:
uuid: "4528d69c95104a80aef5b734b2bce43e"
format_stamp: "Formatted at 2026-08-08 08:25:02 on dist-test-slave-f7th"
server_key: "7bb4c08cfc0adf694503bd04aff26d66"
server_key_iv: "5ec1bd526f28351e8029cda9f421c5c6"
server_key_version: "encryptionkey@0"
I20260808 08:25:02.530905 26318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance:
uuid: "4528d69c95104a80aef5b734b2bce43e"
format_stamp: "Formatted at 2026-08-08 08:25:02 on dist-test-slave-f7th"
server_key: "7bb4c08cfc0adf694503bd04aff26d66"
server_key_iv: "5ec1bd526f28351e8029cda9f421c5c6"
server_key_version: "encryptionkey@0"
I20260808 08:25:02.534576 26318 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 08:25:02.536939 26341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:02.538125 26318 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 08:25:02.538256 26318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-0/wal
uuid: "4528d69c95104a80aef5b734b2bce43e"
format_stamp: "Formatted at 2026-08-08 08:25:02 on dist-test-slave-f7th"
server_key: "7bb4c08cfc0adf694503bd04aff26d66"
server_key_iv: "5ec1bd526f28351e8029cda9f421c5c6"
server_key_version: "encryptionkey@0"
I20260808 08:25:02.538378 26318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:02.551919 26318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:02.555001 26318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:02.555151 26318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:02.556525 26318 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:25:02.556573 26318 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:02.556674 26318 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:25:02.556708 26318 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 08:25:02 dist-test-slave-f7th krb5kdc[26123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177502, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:25:02.567126 26318 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.193:41383
I20260808 08:25:02.567239 26522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.193:41383 every 8 connection(s)
I20260808 08:25:02.568055 26318 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
I20260808 08:25:02.569190 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 26318
I20260808 08:25:02.569326 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance
I20260808 08:25:02.569644 23471 external_mini_cluster.cc:1428] Setting key 519eeaa6d620f5436f29972e85d8474c
I20260808 08:25:02.569736 26284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:02.557261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:40163 (local address 127.22.235.254:36447)
0808 08:25:02.557412 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:02.557417 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:02.558904 (+ 1487us) server_negotiation.cc:407] Connection header received
0808 08:25:02.560239 (+ 1335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:02.560244 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:02.560305 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:02.560394 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:02.561645 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:02.562200 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:02.563379 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:02.563616 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:02.563708 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:02.566112 (+ 2404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:02.566115 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:02.566117 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:02.566143 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:02.567599 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:02.568070 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:02.568073 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:02.568074 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:02.568151 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:02.568561 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:02.568564 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:02.568566 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:02.568834 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:02.568929 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:02.569503 (+ 574us) server_negotiation.cc:299] Negotiation successful
0808 08:25:02.569606 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 08:25:02.570662 26354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:02.558040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:36447 (local address 127.22.235.193:40163)
0808 08:25:02.558545 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:02.558798 (+ 253us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:02.560006 (+ 1208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:02.560585 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:02.560592 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:02.560949 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:02.561469 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:02.561482 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:02.562331 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:02.562334 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:02.563256 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:02.563266 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:02.563768 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:02.563772 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:02.563925 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:02.564476 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:02.564504 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:02.565948 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:02.567702 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:02.567708 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:02.567723 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:02.567970 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:02.568242 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:02.568245 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:02.568247 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:02.568433 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:02.568940 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:02.568946 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:02.569240 (+ 294us) client_negotiation.cc:769] Sending connection context
0808 08:25:02.569455 (+ 215us) client_negotiation.cc:240] Negotiation successful
0808 08:25:02.569788 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":115,"threads_started":1}
I20260808 08:25:02.571987 26524 heartbeater.cc:347] Connected to a master server at 127.22.235.254:36447
I20260808 08:25:02.572227 26524 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:02.572767 26524 heartbeater.cc:511] Master 127.22.235.254:36447 requested a full tablet report, sending...
I20260808 08:25:02.574244 26195 ts_manager.cc:194] Registered new tserver with Master: 4528d69c95104a80aef5b734b2bce43e (127.22.235.193:41383)
I20260808 08:25:02.575593 26195 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.193@KRBTEST.COM'} at 127.22.235.193:40163
I20260808 08:25:02.581988 26533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:02.574923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:36447 (local address 127.22.235.193:58497)
0808 08:25:02.575258 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:02.575270 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:02.575352 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:02.575554 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:02.575556 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:02.575728 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:02.575839 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:02.575845 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:02.576732 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:02.576736 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:02.577562 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:02.577574 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:02.578134 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:02.578137 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:02.578206 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:02.578671 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:02.578682 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:02.579007 (+ 325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:02.580713 (+ 1706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:02.580716 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:02.580717 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:02.580898 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:02.581155 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:02.581157 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:02.581159 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:02.581222 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:02.581586 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:02.581588 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:02.581657 (+ 69us) client_negotiation.cc:769] Sending connection context
0808 08:25:02.581727 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 08:25:02.581828 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":94,"threads_started":1}
I20260808 08:25:02.582078 26284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:02.575015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:58497 (local address 127.22.235.254:36447)
0808 08:25:02.575126 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:02.575131 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:02.575340 (+ 209us) server_negotiation.cc:407] Connection header received
0808 08:25:02.575411 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:02.575415 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:02.575458 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:02.575536 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:02.575937 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:02.576595 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:02.577694 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:02.578018 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:02.578122 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:02.579138 (+ 1016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:02.579142 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:02.579145 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:02.579172 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:02.580598 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:02.580983 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:02.580987 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:02.580989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:02.581079 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:02.581291 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:02.581295 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:02.581297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:02.581505 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:02.581607 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:02.581804 (+ 197us) server_negotiation.cc:299] Negotiation successful
0808 08:25:02.581915 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":39}
WARNING: no policy specified for kudu/127.22.235.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab.
Entry for principal kudu/127.22.235.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab.
I20260808 08:25:02.599056 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.194:0
--local_ip_for_outbound_sockets=127.22.235.194
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:36447
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab
--principal=kudu/127.22.235.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:44915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:25:02.711828 26536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:02.712062 26536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:02.712111 26536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:02.716066 26536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:02.716128 26536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:02.716172 26536 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 08:25:02.716199 26536 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 08:25:02.716249 26536 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.194
I20260808 08:25:02.721630 26536 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:44915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab
--principal=kudu/127.22.235.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.235.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.235.254:36447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26536
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.194
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:02.722635 26536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:02.723639 26536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:02.733139 26543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:02.733193 26536 server_base.cc:1034] running on GCE node
W20260808 08:25:02.733139 26548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:02.733338 26545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:02.733713 26536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:02.734333 26536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:02.735498 26536 hybrid_clock.cc:648] HybridClock initialized: now 1786177502735478 us; error 31 us; skew 500 ppm
Aug 08 08:25:02 dist-test-slave-f7th krb5kdc[26123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177502, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:02.738056 26536 init.cc:377] Logged in from keytab as kudu/127.22.235.194@KRBTEST.COM (short username kudu)
I20260808 08:25:02.738903 26536 webserver.cc:460] Webserver started at http://127.22.235.194:34975/ using document root <none> and password file <none>
I20260808 08:25:02.739435 26536 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:02.739511 26536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:02.739717 26536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:02.741736 26536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-1/data/instance:
uuid: "f49ac766475b4a0990668efe159a6f7b"
format_stamp: "Formatted at 2026-08-08 08:25:02 on dist-test-slave-f7th"
server_key: "f402b45ecd0a9b1f3325f38033add684"
server_key_iv: "b480349ecb7b14d1f6a07b402348bb57"
server_key_version: "encryptionkey@0"
I20260808 08:25:02.742239 26536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance:
uuid: "f49ac766475b4a0990668efe159a6f7b"
format_stamp: "Formatted at 2026-08-08 08:25:02 on dist-test-slave-f7th"
server_key: "f402b45ecd0a9b1f3325f38033add684"
server_key_iv: "b480349ecb7b14d1f6a07b402348bb57"
server_key_version: "encryptionkey@0"
I20260808 08:25:02.745729 26536 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260808 08:25:02.748083 26563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:02.749277 26536 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 08:25:02.749379 26536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-1/wal
uuid: "f49ac766475b4a0990668efe159a6f7b"
format_stamp: "Formatted at 2026-08-08 08:25:02 on dist-test-slave-f7th"
server_key: "f402b45ecd0a9b1f3325f38033add684"
server_key_iv: "b480349ecb7b14d1f6a07b402348bb57"
server_key_version: "encryptionkey@0"
I20260808 08:25:02.749469 26536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:02.768959 26536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:02.771699 26536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:02.771890 26536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:02.773583 26536 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:25:02.773645 26536 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:02.773707 26536 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:25:02.773734 26536 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 08:25:02 dist-test-slave-f7th krb5kdc[26123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177502, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.194@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:25:02.784382 26536 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.194:43833
I20260808 08:25:02.784832 26750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.194:43833 every 8 connection(s)
I20260808 08:25:02.785426 26536 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
I20260808 08:25:02.787132 26284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:02.774492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:40733 (local address 127.22.235.254:36447)
0808 08:25:02.774635 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:02.774640 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:02.775655 (+ 1015us) server_negotiation.cc:407] Connection header received
0808 08:25:02.776590 (+ 935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:02.776594 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:02.776683 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:02.776743 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:02.778221 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:02.778776 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:02.780016 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:02.780316 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:02.780414 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:02.783287 (+ 2873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:02.783292 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:02.783295 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:02.783328 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:02.784924 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:02.785450 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:02.785452 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:02.785454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:02.785565 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:02.785941 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:02.785945 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:02.785947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:02.786204 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:02.786301 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:02.786832 (+ 531us) server_negotiation.cc:299] Negotiation successful
0808 08:25:02.786957 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 08:25:02.787883 26574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:02.775029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:36447 (local address 127.22.235.194:40733)
0808 08:25:02.775507 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:02.775549 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:02.776357 (+ 808us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:02.777173 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:02.777182 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:02.777546 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:02.778064 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:02.778085 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:02.778902 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:02.778905 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:02.779892 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:02.779903 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:02.780477 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:02.780481 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:02.780715 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:02.781325 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:02.781345 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:02.783093 (+ 1748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:02.785029 (+ 1936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:02.785035 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:02.785048 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:02.785356 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:02.785638 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:02.785641 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:02.785643 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:02.785819 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:02.786322 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:02.786329 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:02.786588 (+ 259us) client_negotiation.cc:769] Sending connection context
0808 08:25:02.786761 (+ 173us) client_negotiation.cc:240] Negotiation successful
0808 08:25:02.787088 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":123,"threads_started":1}
I20260808 08:25:02.789014 26751 heartbeater.cc:347] Connected to a master server at 127.22.235.254:36447
I20260808 08:25:02.789161 26751 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:02.789705 26751 heartbeater.cc:511] Master 127.22.235.254:36447 requested a full tablet report, sending...
I20260808 08:25:02.790829 26194 ts_manager.cc:194] Registered new tserver with Master: f49ac766475b4a0990668efe159a6f7b (127.22.235.194:43833)
I20260808 08:25:02.791548 26194 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.194@KRBTEST.COM'} at 127.22.235.194:40733
I20260808 08:25:02.795385 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 26536
I20260808 08:25:02.795512 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance
I20260808 08:25:02.795789 23471 external_mini_cluster.cc:1428] Setting key de289e74e720b135190fd9aa1987fcae
I20260808 08:25:02.798079 26284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:02.791340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:48479 (local address 127.22.235.254:36447)
0808 08:25:02.791454 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:02.791458 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:02.791953 (+ 495us) server_negotiation.cc:407] Connection header received
0808 08:25:02.792077 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:02.792080 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:02.792119 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:02.792179 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:02.792690 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:02.793171 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:02.793955 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:02.794149 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:02.794214 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:02.795132 (+ 918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:02.795135 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:02.795136 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:02.795151 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:02.796517 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:02.796951 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:02.796953 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:02.796954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:02.797033 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:02.797286 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:02.797289 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:02.797291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:02.797505 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:02.797593 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:02.797833 (+ 240us) server_negotiation.cc:299] Negotiation successful
0808 08:25:02.797934 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 08:25:02.798102 26759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:02.791261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:36447 (local address 127.22.235.194:48479)
0808 08:25:02.791867 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:02.791879 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:02.791978 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:02.792216 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:02.792219 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:02.792429 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:02.792548 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:02.792555 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:02.793288 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:02.793292 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:02.793870 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:02.793880 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:02.794222 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:02.794224 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:02.794287 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:02.794726 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:02.794738 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:02.795039 (+ 301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:02.796608 (+ 1569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:02.796611 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:02.796634 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:02.796870 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:02.797115 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:02.797118 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:02.797120 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:02.797198 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:02.797599 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:02.797602 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:02.797681 (+ 79us) client_negotiation.cc:769] Sending connection context
0808 08:25:02.797767 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 08:25:02.797904 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":95,"threads_started":1}
WARNING: no policy specified for kudu/127.22.235.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab.
Entry for principal kudu/127.22.235.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab.
I20260808 08:25:02.821600 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.195:0
--local_ip_for_outbound_sockets=127.22.235.195
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:36447
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab
--principal=kudu/127.22.235.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:44915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:25:02.934556 26764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:02.934803 26764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:02.934847 26764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:02.938763 26764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:02.938822 26764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:02.938859 26764 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 08:25:02.938876 26764 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 08:25:02.938912 26764 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.195
I20260808 08:25:02.944054 26764 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:44915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab
--principal=kudu/127.22.235.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.235.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.235.254:36447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.195
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:02.945055 26764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:02.946022 26764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:02.956044 26774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:02.956037 26770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:02.956037 26782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:02.956424 26764 server_base.cc:1034] running on GCE node
I20260808 08:25:02.956863 26764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:02.957521 26764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:02.958689 26764 hybrid_clock.cc:648] HybridClock initialized: now 1786177502958672 us; error 33 us; skew 500 ppm
Aug 08 08:25:02 dist-test-slave-f7th krb5kdc[26123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177502, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:02.961308 26764 init.cc:377] Logged in from keytab as kudu/127.22.235.195@KRBTEST.COM (short username kudu)
I20260808 08:25:02.962138 26764 webserver.cc:460] Webserver started at http://127.22.235.195:35347/ using document root <none> and password file <none>
I20260808 08:25:02.962657 26764 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:02.962714 26764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:02.962883 26764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:02.964926 26764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-2/data/instance:
uuid: "e454ff6295c54d31877c4683dc9a639b"
format_stamp: "Formatted at 2026-08-08 08:25:02 on dist-test-slave-f7th"
server_key: "77ba89dc9285d828fd21103673d65b5c"
server_key_iv: "8e6e09dbbae99aafb9ce3aeb54d104f7"
server_key_version: "encryptionkey@0"
I20260808 08:25:02.965437 26764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance:
uuid: "e454ff6295c54d31877c4683dc9a639b"
format_stamp: "Formatted at 2026-08-08 08:25:02 on dist-test-slave-f7th"
server_key: "77ba89dc9285d828fd21103673d65b5c"
server_key_iv: "8e6e09dbbae99aafb9ce3aeb54d104f7"
server_key_version: "encryptionkey@0"
I20260808 08:25:02.969105 26764 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260808 08:25:02.971372 26793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:02.972597 26764 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:25:02.972730 26764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-2/wal
uuid: "e454ff6295c54d31877c4683dc9a639b"
format_stamp: "Formatted at 2026-08-08 08:25:02 on dist-test-slave-f7th"
server_key: "77ba89dc9285d828fd21103673d65b5c"
server_key_iv: "8e6e09dbbae99aafb9ce3aeb54d104f7"
server_key_version: "encryptionkey@0"
I20260808 08:25:02.972826 26764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:02.990991 26764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:02.994124 26764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:02.994299 26764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:02.995775 26764 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:25:02.995836 26764 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:02.995882 26764 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:25:02.995911 26764 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 08:25:03 dist-test-slave-f7th krb5kdc[26123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177502, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.195@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:25:03.005995 26764 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.195:44217
I20260808 08:25:03.006106 26989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.195:44217 every 8 connection(s)
I20260808 08:25:03.007138 26764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
I20260808 08:25:03.008227 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 26764
I20260808 08:25:03.008356 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance
I20260808 08:25:03.008669 23471 external_mini_cluster.cc:1428] Setting key 5d90a3f6b8aff202d70b3a1c59fc7176
I20260808 08:25:03.010262 26284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:02.996556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:50657 (local address 127.22.235.254:36447)
0808 08:25:02.996698 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:02.996701 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:02.997745 (+ 1044us) server_negotiation.cc:407] Connection header received
0808 08:25:02.998789 (+ 1044us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:02.998794 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:02.998858 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:02.998918 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:03.000585 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.001368 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.003120 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.003334 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.003416 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:03.006356 (+ 2940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:03.006360 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:03.006363 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:03.006395 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:03.008106 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:03.008682 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:03.008686 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:03.008687 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:03.008785 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:03.009126 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:03.009130 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:03.009132 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:03.009314 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:03.009409 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:03.009980 (+ 571us) server_negotiation.cc:299] Negotiation successful
0808 08:25:03.010106 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 08:25:03.011099 26810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:02.997126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:36447 (local address 127.22.235.195:50657)
0808 08:25:02.997599 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:02.997640 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:02.998536 (+ 896us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:02.999371 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:02.999379 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:02.999760 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:03.000437 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:03.000458 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.001553 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.001557 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:03.002988 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:03.003002 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.003509 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.003514 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:03.003717 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:03.004380 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:03.004402 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:03.006183 (+ 1781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:03.008211 (+ 2028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:03.008215 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:03.008226 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:03.008544 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:03.008879 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:03.008883 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:03.008885 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:03.009025 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:03.009413 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:03.009418 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:03.009703 (+ 285us) client_negotiation.cc:769] Sending connection context
0808 08:25:03.009914 (+ 211us) client_negotiation.cc:240] Negotiation successful
0808 08:25:03.010236 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":126,"threads_started":1}
I20260808 08:25:03.012475 26990 heartbeater.cc:347] Connected to a master server at 127.22.235.254:36447
I20260808 08:25:03.012660 26990 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:03.013283 26990 heartbeater.cc:511] Master 127.22.235.254:36447 requested a full tablet report, sending...
I20260808 08:25:03.014333 26194 ts_manager.cc:194] Registered new tserver with Master: e454ff6295c54d31877c4683dc9a639b (127.22.235.195:44217)
I20260808 08:25:03.015045 26194 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.195@KRBTEST.COM'} at 127.22.235.195:50657
I20260808 08:25:03.020881 26284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:03.014656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:42823 (local address 127.22.235.254:36447)
0808 08:25:03.014767 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:03.014771 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:03.015004 (+ 233us) server_negotiation.cc:407] Connection header received
0808 08:25:03.015092 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:03.015094 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:03.015129 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:03.015204 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:03.015624 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.016109 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.016985 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.017200 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.017283 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:03.018396 (+ 1113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:03.018401 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:03.018404 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:03.018430 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:03.019557 (+ 1127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:03.019967 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:03.019969 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:03.019970 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:03.020040 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:03.020257 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:03.020258 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:03.020259 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:03.020408 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:03.020476 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:03.020678 (+ 202us) server_negotiation.cc:299] Negotiation successful
0808 08:25:03.020772 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 08:25:03.020892 26997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:03.014560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:36447 (local address 127.22.235.195:42823)
0808 08:25:03.014897 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:03.014912 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:03.015012 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:03.015224 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:03.015227 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:03.015414 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:03.015510 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:03.015515 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.016228 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.016231 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:03.016872 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:03.016882 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.017297 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.017301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:03.017386 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:03.017833 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:03.017850 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:03.018249 (+ 399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:03.019671 (+ 1422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:03.019674 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:03.019676 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:03.019865 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:03.020116 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:03.020118 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:03.020119 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:03.020195 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:03.020464 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:03.020467 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:03.020537 (+ 70us) client_negotiation.cc:769] Sending connection context
0808 08:25:03.020608 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 08:25:03.020763 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":98,"threads_started":1}
I20260808 08:25:03.023082 23471 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 08:25:03 dist-test-slave-f7th krb5kdc[26123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 08:25:03 dist-test-slave-f7th krb5kdc[26123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:25:03.048439 26284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:03.040956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33236 (local address 127.22.235.254:36447)
0808 08:25:03.041088 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:03.041092 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:03.041104 (+ 12us) server_negotiation.cc:407] Connection header received
0808 08:25:03.041143 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:03.041146 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:03.041184 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:03.041253 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:03.041952 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.042460 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.043276 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.043485 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.043581 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:03.045818 (+ 2237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:03.045821 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:03.045823 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:03.045848 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:03.046941 (+ 1093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:03.047318 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:03.047321 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:03.047323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:03.047396 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:03.047637 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:03.047640 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:03.047642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:03.047814 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:03.047930 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:03.048164 (+ 234us) server_negotiation.cc:299] Negotiation successful
0808 08:25:03.048273 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 08:25:03.051265 26194 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33236:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 08:25:03.054461 26194 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 08:25:03.068377 27015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:03.062488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.195:44217 (local address 127.0.0.1:48792)
0808 08:25:03.062863 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:03.063167 (+ 304us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:03.063304 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:03.064100 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:03.064107 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:03.064133 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:25:03.064265 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:03.064272 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.065536 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.065539 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:03.066986 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:03.067004 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.067839 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.067843 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:03.067955 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:03.068006 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 08:25:03.068110 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 08:25:03.068231 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":37,"thread_start_us":92,"threads_started":1}
I20260808 08:25:03.068389 27021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:03.062655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48792 (local address 127.22.235.195:44217)
0808 08:25:03.063092 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:03.063104 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:03.063447 (+ 343us) server_negotiation.cc:407] Connection header received
0808 08:25:03.063532 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:03.063537 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:03.063754 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:03.063912 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:25:03.064380 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.065417 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.067125 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.067723 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.067852 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:03.067991 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:03.068158 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 08:25:03.068246 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":115,"threads_started":1}
I20260808 08:25:03.068915 27022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:03.063343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48258 (local address 127.22.235.193:41383)
0808 08:25:03.063719 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:03.063735 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:03.064210 (+ 475us) server_negotiation.cc:407] Connection header received
0808 08:25:03.064290 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:03.064295 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:03.064471 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:03.064654 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:25:03.064902 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.065800 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.067167 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.067931 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.068077 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:03.068206 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:03.068692 (+ 486us) server_negotiation.cc:299] Negotiation successful
0808 08:25:03.068778 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":98,"threads_started":1}
I20260808 08:25:03.069167 27025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:03.062605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:41383 (local address 127.0.0.1:48258)
0808 08:25:03.064007 (+ 1402us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:03.064021 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:03.064120 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:03.064672 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:03.064676 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:03.064697 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:25:03.064792 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:03.064798 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.065926 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.065929 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:03.067054 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:03.067063 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.068272 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.068275 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:03.068397 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:03.068456 (+ 59us) client_negotiation.cc:769] Sending connection context
0808 08:25:03.068535 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 08:25:03.068601 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1341,"mutex_wait_us":21,"thread_start_us":104,"threads_started":1}
I20260808 08:25:03.069191 27026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:03.063930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51836 (local address 127.22.235.194:43833)
0808 08:25:03.064359 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:03.064370 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:03.064389 (+ 19us) server_negotiation.cc:407] Connection header received
0808 08:25:03.064452 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:03.064456 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:03.064639 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:03.064770 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:25:03.065142 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.066133 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.067299 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.068034 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.068148 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:03.068246 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:03.068958 (+ 712us) server_negotiation.cc:299] Negotiation successful
0808 08:25:03.069058 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":128,"threads_started":1}
I20260808 08:25:03.069808 27016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:03.063786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.194:43833 (local address 127.0.0.1:51836)
0808 08:25:03.064094 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:03.064107 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:03.064188 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:03.064890 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:03.064893 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:03.064911 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:25:03.065019 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:03.065026 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.066260 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.066263 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:03.067181 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:03.067192 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.068294 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.068298 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:03.068805 (+ 507us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:03.068833 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 08:25:03.068910 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 08:25:03.069549 (+ 639us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"mutex_wait_us":73,"spinlock_wait_cycles":768,"thread_start_us":94,"threads_started":1}
I20260808 08:25:03.071743 26872 tablet_service.cc:1467] Processing CreateTablet for tablet a6d62ca3ff214ef69a97fb2a4039c2f6 (DEFAULT_TABLE table=test-table [id=afc4eb2ef695474b9f8831a1acfc6220]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:25:03.072314 26643 tablet_service.cc:1467] Processing CreateTablet for tablet a6d62ca3ff214ef69a97fb2a4039c2f6 (DEFAULT_TABLE table=test-table [id=afc4eb2ef695474b9f8831a1acfc6220]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:25:03.073179 26872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6d62ca3ff214ef69a97fb2a4039c2f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:03.072885 26415 tablet_service.cc:1467] Processing CreateTablet for tablet a6d62ca3ff214ef69a97fb2a4039c2f6 (DEFAULT_TABLE table=test-table [id=afc4eb2ef695474b9f8831a1acfc6220]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:25:03.073340 26643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6d62ca3ff214ef69a97fb2a4039c2f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:03.073863 26415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6d62ca3ff214ef69a97fb2a4039c2f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:03.079916 27027 tablet_bootstrap.cc:492] T a6d62ca3ff214ef69a97fb2a4039c2f6 P f49ac766475b4a0990668efe159a6f7b: Bootstrap starting.
I20260808 08:25:03.081218 27034 tablet_bootstrap.cc:492] T a6d62ca3ff214ef69a97fb2a4039c2f6 P e454ff6295c54d31877c4683dc9a639b: Bootstrap starting.
I20260808 08:25:03.082193 27027 tablet_bootstrap.cc:654] T a6d62ca3ff214ef69a97fb2a4039c2f6 P f49ac766475b4a0990668efe159a6f7b: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:03.082938 27027 log.cc:824] T a6d62ca3ff214ef69a97fb2a4039c2f6 P f49ac766475b4a0990668efe159a6f7b: Log is configured to *not* fsync() on all Append() calls
I20260808 08:25:03.083022 27034 tablet_bootstrap.cc:654] T a6d62ca3ff214ef69a97fb2a4039c2f6 P e454ff6295c54d31877c4683dc9a639b: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:03.083582 27030 tablet_bootstrap.cc:492] T a6d62ca3ff214ef69a97fb2a4039c2f6 P 4528d69c95104a80aef5b734b2bce43e: Bootstrap starting.
I20260808 08:25:03.083858 27034 log.cc:824] T a6d62ca3ff214ef69a97fb2a4039c2f6 P e454ff6295c54d31877c4683dc9a639b: Log is configured to *not* fsync() on all Append() calls
I20260808 08:25:03.085215 27027 tablet_bootstrap.cc:492] T a6d62ca3ff214ef69a97fb2a4039c2f6 P f49ac766475b4a0990668efe159a6f7b: No bootstrap required, opened a new log
I20260808 08:25:03.085408 27027 ts_tablet_manager.cc:1397] T a6d62ca3ff214ef69a97fb2a4039c2f6 P f49ac766475b4a0990668efe159a6f7b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 08:25:03.085592 27030 tablet_bootstrap.cc:654] T a6d62ca3ff214ef69a97fb2a4039c2f6 P 4528d69c95104a80aef5b734b2bce43e: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:03.085839 27034 tablet_bootstrap.cc:492] T a6d62ca3ff214ef69a97fb2a4039c2f6 P e454ff6295c54d31877c4683dc9a639b: No bootstrap required, opened a new log
I20260808 08:25:03.086047 27034 ts_tablet_manager.cc:1397] T a6d62ca3ff214ef69a97fb2a4039c2f6 P e454ff6295c54d31877c4683dc9a639b: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 08:25:03.086757 27030 log.cc:824] T a6d62ca3ff214ef69a97fb2a4039c2f6 P 4528d69c95104a80aef5b734b2bce43e: Log is configured to *not* fsync() on all Append() calls
I20260808 08:25:03.088672 27030 tablet_bootstrap.cc:492] T a6d62ca3ff214ef69a97fb2a4039c2f6 P 4528d69c95104a80aef5b734b2bce43e: No bootstrap required, opened a new log
I20260808 08:25:03.088544 27027 raft_consensus.cc:348] T a6d62ca3ff214ef69a97fb2a4039c2f6 P f49ac766475b4a0990668efe159a6f7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } } peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } }
I20260808 08:25:03.088821 27027 raft_consensus.cc:374] T a6d62ca3ff214ef69a97fb2a4039c2f6 P f49ac766475b4a0990668efe159a6f7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:03.088847 27030 ts_tablet_manager.cc:1397] T a6d62ca3ff214ef69a97fb2a4039c2f6 P 4528d69c95104a80aef5b734b2bce43e: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260808 08:25:03.088877 27027 raft_consensus.cc:729] T a6d62ca3ff214ef69a97fb2a4039c2f6 P f49ac766475b4a0990668efe159a6f7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f49ac766475b4a0990668efe159a6f7b, State: Initialized, Role: FOLLOWER
I20260808 08:25:03.089300 27027 consensus_queue.cc:260] T a6d62ca3ff214ef69a97fb2a4039c2f6 P f49ac766475b4a0990668efe159a6f7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } } peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } }
I20260808 08:25:03.089753 27034 raft_consensus.cc:348] T a6d62ca3ff214ef69a97fb2a4039c2f6 P e454ff6295c54d31877c4683dc9a639b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } } peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } }
I20260808 08:25:03.089985 27034 raft_consensus.cc:374] T a6d62ca3ff214ef69a97fb2a4039c2f6 P e454ff6295c54d31877c4683dc9a639b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:03.090050 27034 raft_consensus.cc:729] T a6d62ca3ff214ef69a97fb2a4039c2f6 P e454ff6295c54d31877c4683dc9a639b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e454ff6295c54d31877c4683dc9a639b, State: Initialized, Role: FOLLOWER
I20260808 08:25:03.090180 27027 ts_tablet_manager.cc:1428] T a6d62ca3ff214ef69a97fb2a4039c2f6 P f49ac766475b4a0990668efe159a6f7b: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 08:25:03.090541 26751 heartbeater.cc:503] Master 127.22.235.254:36447 was elected leader, sending a full tablet report...
I20260808 08:25:03.090912 27034 consensus_queue.cc:260] T a6d62ca3ff214ef69a97fb2a4039c2f6 P e454ff6295c54d31877c4683dc9a639b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } } peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } }
I20260808 08:25:03.091837 27034 ts_tablet_manager.cc:1428] T a6d62ca3ff214ef69a97fb2a4039c2f6 P e454ff6295c54d31877c4683dc9a639b: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 08:25:03.091841 26990 heartbeater.cc:503] Master 127.22.235.254:36447 was elected leader, sending a full tablet report...
I20260808 08:25:03.092360 27030 raft_consensus.cc:348] T a6d62ca3ff214ef69a97fb2a4039c2f6 P 4528d69c95104a80aef5b734b2bce43e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } } peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } }
I20260808 08:25:03.092536 27030 raft_consensus.cc:374] T a6d62ca3ff214ef69a97fb2a4039c2f6 P 4528d69c95104a80aef5b734b2bce43e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:03.092566 27030 raft_consensus.cc:729] T a6d62ca3ff214ef69a97fb2a4039c2f6 P 4528d69c95104a80aef5b734b2bce43e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4528d69c95104a80aef5b734b2bce43e, State: Initialized, Role: FOLLOWER
I20260808 08:25:03.092979 27030 consensus_queue.cc:260] T a6d62ca3ff214ef69a97fb2a4039c2f6 P 4528d69c95104a80aef5b734b2bce43e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } } peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } }
I20260808 08:25:03.093611 26524 heartbeater.cc:503] Master 127.22.235.254:36447 was elected leader, sending a full tablet report...
I20260808 08:25:03.093946 27030 ts_tablet_manager.cc:1428] T a6d62ca3ff214ef69a97fb2a4039c2f6 P 4528d69c95104a80aef5b734b2bce43e: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260808 08:25:03.258165 26991 tablet.cc:2369] T a6d62ca3ff214ef69a97fb2a4039c2f6 P e454ff6295c54d31877c4683dc9a639b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 08:25:03.283114 27039 raft_consensus.cc:482] T a6d62ca3ff214ef69a97fb2a4039c2f6 P f49ac766475b4a0990668efe159a6f7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 08:25:03.283318 27039 raft_consensus.cc:504] T a6d62ca3ff214ef69a97fb2a4039c2f6 P f49ac766475b4a0990668efe159a6f7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } } peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } }
I20260808 08:25:03.284554 27039 leader_election.cc:288] T a6d62ca3ff214ef69a97fb2a4039c2f6 P f49ac766475b4a0990668efe159a6f7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e454ff6295c54d31877c4683dc9a639b (127.22.235.195:44217), 4528d69c95104a80aef5b734b2bce43e (127.22.235.193:41383)
W20260808 08:25:03.286636 26753 tablet.cc:2369] T a6d62ca3ff214ef69a97fb2a4039c2f6 P f49ac766475b4a0990668efe159a6f7b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 08:25:03.289770 26574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:03.284780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.195:44217 (local address 127.22.235.194:32811)
0808 08:25:03.284920 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:03.284937 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:03.285070 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:03.285675 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:03.285678 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:03.285701 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:25:03.285819 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:03.285825 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.287180 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.287183 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:03.288353 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:03.288363 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.289326 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.289330 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:03.289443 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:03.289476 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 08:25:03.289565 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 08:25:03.289640 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 08:25:03.289772 27021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:03.285344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:32811 (local address 127.22.235.195:44217)
0808 08:25:03.285464 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:03.285468 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:03.285481 (+ 13us) server_negotiation.cc:407] Connection header received
0808 08:25:03.285531 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:03.285534 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:03.285573 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:03.285632 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:25:03.285916 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.286428 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.288493 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.289213 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.289335 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:03.289389 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:03.289572 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 08:25:03.289643 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 08:25:03.290495 26907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6d62ca3ff214ef69a97fb2a4039c2f6" candidate_uuid: "f49ac766475b4a0990668efe159a6f7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e454ff6295c54d31877c4683dc9a639b" is_pre_election: true
I20260808 08:25:03.290831 26907 raft_consensus.cc:2454] T a6d62ca3ff214ef69a97fb2a4039c2f6 P e454ff6295c54d31877c4683dc9a639b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f49ac766475b4a0990668efe159a6f7b in term 0.
I20260808 08:25:03.291252 27044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:03.284779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:41383 (local address 127.22.235.194:43981)
0808 08:25:03.286974 (+ 2195us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:03.286987 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:03.287088 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:03.287301 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:03.287304 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:03.287319 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:25:03.287447 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:03.287452 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.288164 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.288167 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:03.289855 (+ 1688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:03.289866 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.290815 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.290819 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:03.290913 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:03.290935 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 08:25:03.291044 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 08:25:03.291111 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2134,"mutex_wait_us":189,"thread_start_us":78,"threads_started":1}
I20260808 08:25:03.291291 27022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:03.284966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:43981 (local address 127.22.235.193:41383)
0808 08:25:03.285087 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:03.285093 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:03.287080 (+ 1987us) server_negotiation.cc:407] Connection header received
0808 08:25:03.287173 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:03.287176 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:03.287222 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:03.287283 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:25:03.287554 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.288063 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.290009 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.290561 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.290658 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:03.290704 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:03.291084 (+ 380us) server_negotiation.cc:299] Negotiation successful
0808 08:25:03.291173 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 08:25:03.291358 26569 leader_election.cc:302] T a6d62ca3ff214ef69a97fb2a4039c2f6 P f49ac766475b4a0990668efe159a6f7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e454ff6295c54d31877c4683dc9a639b, f49ac766475b4a0990668efe159a6f7b; no voters:
I20260808 08:25:03.291728 27039 raft_consensus.cc:2781] T a6d62ca3ff214ef69a97fb2a4039c2f6 P f49ac766475b4a0990668efe159a6f7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 08:25:03.291797 27039 raft_consensus.cc:482] T a6d62ca3ff214ef69a97fb2a4039c2f6 P f49ac766475b4a0990668efe159a6f7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 08:25:03.291843 27039 raft_consensus.cc:3037] T a6d62ca3ff214ef69a97fb2a4039c2f6 P f49ac766475b4a0990668efe159a6f7b [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:03.292048 26445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6d62ca3ff214ef69a97fb2a4039c2f6" candidate_uuid: "f49ac766475b4a0990668efe159a6f7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4528d69c95104a80aef5b734b2bce43e" is_pre_election: true
I20260808 08:25:03.292315 26445 raft_consensus.cc:2454] T a6d62ca3ff214ef69a97fb2a4039c2f6 P 4528d69c95104a80aef5b734b2bce43e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f49ac766475b4a0990668efe159a6f7b in term 0.
I20260808 08:25:03.293032 27039 raft_consensus.cc:504] T a6d62ca3ff214ef69a97fb2a4039c2f6 P f49ac766475b4a0990668efe159a6f7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } } peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } }
I20260808 08:25:03.293385 27039 leader_election.cc:288] T a6d62ca3ff214ef69a97fb2a4039c2f6 P f49ac766475b4a0990668efe159a6f7b [CANDIDATE]: Term 1 election: Requested vote from peers e454ff6295c54d31877c4683dc9a639b (127.22.235.195:44217), 4528d69c95104a80aef5b734b2bce43e (127.22.235.193:41383)
I20260808 08:25:03.293723 26907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6d62ca3ff214ef69a97fb2a4039c2f6" candidate_uuid: "f49ac766475b4a0990668efe159a6f7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e454ff6295c54d31877c4683dc9a639b"
I20260808 08:25:03.293848 26907 raft_consensus.cc:3037] T a6d62ca3ff214ef69a97fb2a4039c2f6 P e454ff6295c54d31877c4683dc9a639b [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:03.293928 26445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6d62ca3ff214ef69a97fb2a4039c2f6" candidate_uuid: "f49ac766475b4a0990668efe159a6f7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4528d69c95104a80aef5b734b2bce43e"
I20260808 08:25:03.294039 26445 raft_consensus.cc:3037] T a6d62ca3ff214ef69a97fb2a4039c2f6 P 4528d69c95104a80aef5b734b2bce43e [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:03.295123 26907 raft_consensus.cc:2454] T a6d62ca3ff214ef69a97fb2a4039c2f6 P e454ff6295c54d31877c4683dc9a639b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f49ac766475b4a0990668efe159a6f7b in term 1.
I20260808 08:25:03.295245 26445 raft_consensus.cc:2454] T a6d62ca3ff214ef69a97fb2a4039c2f6 P 4528d69c95104a80aef5b734b2bce43e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f49ac766475b4a0990668efe159a6f7b in term 1.
I20260808 08:25:03.295440 26569 leader_election.cc:302] T a6d62ca3ff214ef69a97fb2a4039c2f6 P f49ac766475b4a0990668efe159a6f7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e454ff6295c54d31877c4683dc9a639b, f49ac766475b4a0990668efe159a6f7b; no voters:
I20260808 08:25:03.295706 27039 raft_consensus.cc:2781] T a6d62ca3ff214ef69a97fb2a4039c2f6 P f49ac766475b4a0990668efe159a6f7b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:25:03.295897 27039 raft_consensus.cc:686] T a6d62ca3ff214ef69a97fb2a4039c2f6 P f49ac766475b4a0990668efe159a6f7b [term 1 LEADER]: Becoming Leader. State: Replica: f49ac766475b4a0990668efe159a6f7b, State: Running, Role: LEADER
I20260808 08:25:03.296206 27039 consensus_queue.cc:237] T a6d62ca3ff214ef69a97fb2a4039c2f6 P f49ac766475b4a0990668efe159a6f7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } } peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } }
I20260808 08:25:03.299403 26194 catalog_manager.cc:5495] T a6d62ca3ff214ef69a97fb2a4039c2f6 P f49ac766475b4a0990668efe159a6f7b reported cstate change: term changed from 0 to 1, leader changed from <none> to f49ac766475b4a0990668efe159a6f7b (127.22.235.194). New cstate: current_term: 1 leader_uuid: "f49ac766475b4a0990668efe159a6f7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } health_report { overall_health: HEALTHY } } }
W20260808 08:25:03.319522 26526 tablet.cc:2369] T a6d62ca3ff214ef69a97fb2a4039c2f6 P 4528d69c95104a80aef5b734b2bce43e: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 08:25:03 dist-test-slave-f7th krb5kdc[26123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177502, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:25:03.390058 27057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:03.382367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:36447 (local address 127.0.0.1:33242)
0808 08:25:03.382666 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:03.382678 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:03.382791 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:03.383025 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:03.383028 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:03.383439 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:03.383564 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:03.383572 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.384290 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.384293 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:03.384858 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:03.384867 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.385294 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.385297 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:03.385385 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:03.385919 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:03.385934 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:03.387343 (+ 1409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:03.388681 (+ 1338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:03.388690 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:03.388696 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:03.388919 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:03.389157 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:03.389159 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:03.389160 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:03.389241 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:03.389563 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:03.389568 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:03.389681 (+ 113us) client_negotiation.cc:769] Sending connection context
0808 08:25:03.389773 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 08:25:03.389889 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":104,"threads_started":1}
I20260808 08:25:03.390089 26284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:03.382416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33242 (local address 127.22.235.254:36447)
0808 08:25:03.382548 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:03.382552 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:03.382756 (+ 204us) server_negotiation.cc:407] Connection header received
0808 08:25:03.382880 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:03.382883 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:03.382922 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:03.382987 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:03.383681 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.384194 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.384974 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.385196 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.385280 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:03.387470 (+ 2190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:03.387473 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:03.387475 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:03.387500 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:03.388567 (+ 1067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:03.389019 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:03.389022 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:03.389023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:03.389091 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:03.389303 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:03.389305 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:03.389306 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:03.389474 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:03.389548 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:03.389816 (+ 268us) server_negotiation.cc:299] Negotiation successful
0808 08:25:03.389942 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":45}
I20260808 08:25:03.393767 26194 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:33242:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 08:25:03.394495 26194 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 08:25:03.399183 26643 tablet_service.cc:1467] Processing CreateTablet for tablet 134f85d1da2f4127ad90c2a3803f6842 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2a530621b0de41deb7b63218bd5dae6e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 08:25:03.399253 26872 tablet_service.cc:1467] Processing CreateTablet for tablet 134f85d1da2f4127ad90c2a3803f6842 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2a530621b0de41deb7b63218bd5dae6e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 08:25:03.399104 26415 tablet_service.cc:1467] Processing CreateTablet for tablet 134f85d1da2f4127ad90c2a3803f6842 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2a530621b0de41deb7b63218bd5dae6e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 08:25:03.399854 26643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 134f85d1da2f4127ad90c2a3803f6842. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:03.399856 26872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 134f85d1da2f4127ad90c2a3803f6842. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:03.399856 26415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 134f85d1da2f4127ad90c2a3803f6842. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:03.402448 27027 tablet_bootstrap.cc:492] T 134f85d1da2f4127ad90c2a3803f6842 P f49ac766475b4a0990668efe159a6f7b: Bootstrap starting.
I20260808 08:25:03.402691 27030 tablet_bootstrap.cc:492] T 134f85d1da2f4127ad90c2a3803f6842 P 4528d69c95104a80aef5b734b2bce43e: Bootstrap starting.
I20260808 08:25:03.402766 27034 tablet_bootstrap.cc:492] T 134f85d1da2f4127ad90c2a3803f6842 P e454ff6295c54d31877c4683dc9a639b: Bootstrap starting.
I20260808 08:25:03.403501 27030 tablet_bootstrap.cc:654] T 134f85d1da2f4127ad90c2a3803f6842 P 4528d69c95104a80aef5b734b2bce43e: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:03.403518 27027 tablet_bootstrap.cc:654] T 134f85d1da2f4127ad90c2a3803f6842 P f49ac766475b4a0990668efe159a6f7b: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:03.403677 27034 tablet_bootstrap.cc:654] T 134f85d1da2f4127ad90c2a3803f6842 P e454ff6295c54d31877c4683dc9a639b: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:03.404726 27030 tablet_bootstrap.cc:492] T 134f85d1da2f4127ad90c2a3803f6842 P 4528d69c95104a80aef5b734b2bce43e: No bootstrap required, opened a new log
I20260808 08:25:03.404819 27030 ts_tablet_manager.cc:1397] T 134f85d1da2f4127ad90c2a3803f6842 P 4528d69c95104a80aef5b734b2bce43e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 08:25:03.405014 27027 tablet_bootstrap.cc:492] T 134f85d1da2f4127ad90c2a3803f6842 P f49ac766475b4a0990668efe159a6f7b: No bootstrap required, opened a new log
I20260808 08:25:03.405006 27034 tablet_bootstrap.cc:492] T 134f85d1da2f4127ad90c2a3803f6842 P e454ff6295c54d31877c4683dc9a639b: No bootstrap required, opened a new log
I20260808 08:25:03.405079 27034 ts_tablet_manager.cc:1397] T 134f85d1da2f4127ad90c2a3803f6842 P e454ff6295c54d31877c4683dc9a639b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 08:25:03.405079 27027 ts_tablet_manager.cc:1397] T 134f85d1da2f4127ad90c2a3803f6842 P f49ac766475b4a0990668efe159a6f7b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 08:25:03.405206 27030 raft_consensus.cc:348] T 134f85d1da2f4127ad90c2a3803f6842 P 4528d69c95104a80aef5b734b2bce43e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } }
I20260808 08:25:03.405305 27030 raft_consensus.cc:374] T 134f85d1da2f4127ad90c2a3803f6842 P 4528d69c95104a80aef5b734b2bce43e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:03.405326 27030 raft_consensus.cc:729] T 134f85d1da2f4127ad90c2a3803f6842 P 4528d69c95104a80aef5b734b2bce43e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4528d69c95104a80aef5b734b2bce43e, State: Initialized, Role: FOLLOWER
I20260808 08:25:03.405431 27030 consensus_queue.cc:260] T 134f85d1da2f4127ad90c2a3803f6842 P 4528d69c95104a80aef5b734b2bce43e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } }
I20260808 08:25:03.405512 27034 raft_consensus.cc:348] T 134f85d1da2f4127ad90c2a3803f6842 P e454ff6295c54d31877c4683dc9a639b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } }
I20260808 08:25:03.405503 27027 raft_consensus.cc:348] T 134f85d1da2f4127ad90c2a3803f6842 P f49ac766475b4a0990668efe159a6f7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } }
I20260808 08:25:03.405606 27034 raft_consensus.cc:374] T 134f85d1da2f4127ad90c2a3803f6842 P e454ff6295c54d31877c4683dc9a639b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:03.405606 27027 raft_consensus.cc:374] T 134f85d1da2f4127ad90c2a3803f6842 P f49ac766475b4a0990668efe159a6f7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:03.405637 27034 raft_consensus.cc:729] T 134f85d1da2f4127ad90c2a3803f6842 P e454ff6295c54d31877c4683dc9a639b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e454ff6295c54d31877c4683dc9a639b, State: Initialized, Role: FOLLOWER
I20260808 08:25:03.405637 27027 raft_consensus.cc:729] T 134f85d1da2f4127ad90c2a3803f6842 P f49ac766475b4a0990668efe159a6f7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f49ac766475b4a0990668efe159a6f7b, State: Initialized, Role: FOLLOWER
I20260808 08:25:03.405680 27030 ts_tablet_manager.cc:1428] T 134f85d1da2f4127ad90c2a3803f6842 P 4528d69c95104a80aef5b734b2bce43e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 08:25:03.405750 27027 consensus_queue.cc:260] T 134f85d1da2f4127ad90c2a3803f6842 P f49ac766475b4a0990668efe159a6f7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } }
I20260808 08:25:03.405750 27034 consensus_queue.cc:260] T 134f85d1da2f4127ad90c2a3803f6842 P e454ff6295c54d31877c4683dc9a639b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } }
I20260808 08:25:03.405686 27041 tablet_replica.cc:439] T 134f85d1da2f4127ad90c2a3803f6842 P 4528d69c95104a80aef5b734b2bce43e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } } }
I20260808 08:25:03.405906 27041 tablet_replica.cc:442] T 134f85d1da2f4127ad90c2a3803f6842 P 4528d69c95104a80aef5b734b2bce43e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 08:25:03.405987 27034 ts_tablet_manager.cc:1428] T 134f85d1da2f4127ad90c2a3803f6842 P e454ff6295c54d31877c4683dc9a639b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 08:25:03.405987 27027 ts_tablet_manager.cc:1428] T 134f85d1da2f4127ad90c2a3803f6842 P f49ac766475b4a0990668efe159a6f7b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 08:25:03.406243 27040 tablet_replica.cc:439] T 134f85d1da2f4127ad90c2a3803f6842 P e454ff6295c54d31877c4683dc9a639b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } } }
I20260808 08:25:03.406380 27040 tablet_replica.cc:442] T 134f85d1da2f4127ad90c2a3803f6842 P e454ff6295c54d31877c4683dc9a639b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 08:25:03.406561 27039 tablet_replica.cc:439] T 134f85d1da2f4127ad90c2a3803f6842 P f49ac766475b4a0990668efe159a6f7b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } } }
I20260808 08:25:03.406724 27039 tablet_replica.cc:442] T 134f85d1da2f4127ad90c2a3803f6842 P f49ac766475b4a0990668efe159a6f7b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 08:25:03.446453 27039 raft_consensus.cc:482] T 134f85d1da2f4127ad90c2a3803f6842 P f49ac766475b4a0990668efe159a6f7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 08:25:03.446588 27039 raft_consensus.cc:504] T 134f85d1da2f4127ad90c2a3803f6842 P f49ac766475b4a0990668efe159a6f7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } }
I20260808 08:25:03.447029 27039 leader_election.cc:288] T 134f85d1da2f4127ad90c2a3803f6842 P f49ac766475b4a0990668efe159a6f7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4528d69c95104a80aef5b734b2bce43e (127.22.235.193:41383), e454ff6295c54d31877c4683dc9a639b (127.22.235.195:44217)
I20260808 08:25:03.447403 26445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "134f85d1da2f4127ad90c2a3803f6842" candidate_uuid: "f49ac766475b4a0990668efe159a6f7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4528d69c95104a80aef5b734b2bce43e" is_pre_election: true
I20260808 08:25:03.447466 26907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "134f85d1da2f4127ad90c2a3803f6842" candidate_uuid: "f49ac766475b4a0990668efe159a6f7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e454ff6295c54d31877c4683dc9a639b" is_pre_election: true
I20260808 08:25:03.447561 26445 raft_consensus.cc:2454] T 134f85d1da2f4127ad90c2a3803f6842 P 4528d69c95104a80aef5b734b2bce43e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f49ac766475b4a0990668efe159a6f7b in term 0.
I20260808 08:25:03.447606 26907 raft_consensus.cc:2454] T 134f85d1da2f4127ad90c2a3803f6842 P e454ff6295c54d31877c4683dc9a639b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f49ac766475b4a0990668efe159a6f7b in term 0.
I20260808 08:25:03.447875 26565 leader_election.cc:302] T 134f85d1da2f4127ad90c2a3803f6842 P f49ac766475b4a0990668efe159a6f7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4528d69c95104a80aef5b734b2bce43e, f49ac766475b4a0990668efe159a6f7b; no voters:
I20260808 08:25:03.448067 27039 raft_consensus.cc:2781] T 134f85d1da2f4127ad90c2a3803f6842 P f49ac766475b4a0990668efe159a6f7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 08:25:03.448136 27039 raft_consensus.cc:482] T 134f85d1da2f4127ad90c2a3803f6842 P f49ac766475b4a0990668efe159a6f7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 08:25:03.448174 27039 raft_consensus.cc:3037] T 134f85d1da2f4127ad90c2a3803f6842 P f49ac766475b4a0990668efe159a6f7b [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:03.449116 27039 raft_consensus.cc:504] T 134f85d1da2f4127ad90c2a3803f6842 P f49ac766475b4a0990668efe159a6f7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } }
I20260808 08:25:03.449410 27039 leader_election.cc:288] T 134f85d1da2f4127ad90c2a3803f6842 P f49ac766475b4a0990668efe159a6f7b [CANDIDATE]: Term 1 election: Requested vote from peers 4528d69c95104a80aef5b734b2bce43e (127.22.235.193:41383), e454ff6295c54d31877c4683dc9a639b (127.22.235.195:44217)
I20260808 08:25:03.449761 26445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "134f85d1da2f4127ad90c2a3803f6842" candidate_uuid: "f49ac766475b4a0990668efe159a6f7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4528d69c95104a80aef5b734b2bce43e"
I20260808 08:25:03.449805 26907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "134f85d1da2f4127ad90c2a3803f6842" candidate_uuid: "f49ac766475b4a0990668efe159a6f7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e454ff6295c54d31877c4683dc9a639b"
I20260808 08:25:03.449913 26445 raft_consensus.cc:3037] T 134f85d1da2f4127ad90c2a3803f6842 P 4528d69c95104a80aef5b734b2bce43e [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:03.449911 26907 raft_consensus.cc:3037] T 134f85d1da2f4127ad90c2a3803f6842 P e454ff6295c54d31877c4683dc9a639b [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:03.451005 26907 raft_consensus.cc:2454] T 134f85d1da2f4127ad90c2a3803f6842 P e454ff6295c54d31877c4683dc9a639b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f49ac766475b4a0990668efe159a6f7b in term 1.
I20260808 08:25:03.450996 26445 raft_consensus.cc:2454] T 134f85d1da2f4127ad90c2a3803f6842 P 4528d69c95104a80aef5b734b2bce43e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f49ac766475b4a0990668efe159a6f7b in term 1.
I20260808 08:25:03.451259 26569 leader_election.cc:302] T 134f85d1da2f4127ad90c2a3803f6842 P f49ac766475b4a0990668efe159a6f7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e454ff6295c54d31877c4683dc9a639b, f49ac766475b4a0990668efe159a6f7b; no voters:
I20260808 08:25:03.451426 27039 raft_consensus.cc:2781] T 134f85d1da2f4127ad90c2a3803f6842 P f49ac766475b4a0990668efe159a6f7b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:25:03.451542 27039 raft_consensus.cc:686] T 134f85d1da2f4127ad90c2a3803f6842 P f49ac766475b4a0990668efe159a6f7b [term 1 LEADER]: Becoming Leader. State: Replica: f49ac766475b4a0990668efe159a6f7b, State: Running, Role: LEADER
I20260808 08:25:03.451680 27039 consensus_queue.cc:237] T 134f85d1da2f4127ad90c2a3803f6842 P f49ac766475b4a0990668efe159a6f7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } }
I20260808 08:25:03.452368 27046 tablet_replica.cc:439] T 134f85d1da2f4127ad90c2a3803f6842 P f49ac766475b4a0990668efe159a6f7b: TxnStatusTablet state changed. Reason: New leader f49ac766475b4a0990668efe159a6f7b. Latest consensus state: current_term: 1 leader_uuid: "f49ac766475b4a0990668efe159a6f7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } } }
I20260808 08:25:03.452486 27046 tablet_replica.cc:442] T 134f85d1da2f4127ad90c2a3803f6842 P f49ac766475b4a0990668efe159a6f7b: This TxnStatusTablet replica's current role is: LEADER
I20260808 08:25:03.452831 27064 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 08:25:03.453081 27064 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 08:25:03.453426 26195 catalog_manager.cc:5495] T 134f85d1da2f4127ad90c2a3803f6842 P f49ac766475b4a0990668efe159a6f7b reported cstate change: term changed from 0 to 1, leader changed from <none> to f49ac766475b4a0990668efe159a6f7b (127.22.235.194). New cstate: current_term: 1 leader_uuid: "f49ac766475b4a0990668efe159a6f7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } health_report { overall_health: UNKNOWN } } }
I20260808 08:25:03.473007 27026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:03.469207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51840 (local address 127.22.235.194:43833)
0808 08:25:03.469315 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:03.469318 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:03.469409 (+ 91us) server_negotiation.cc:407] Connection header received
0808 08:25:03.469601 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:03.469603 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:03.469654 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:03.469732 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:25:03.469938 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.470423 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.471580 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.471788 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.471872 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:03.472167 (+ 295us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:25:03.472604 (+ 437us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 08:25:03.472718 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:03.472842 (+ 124us) server_negotiation.cc:299] Negotiation successful
0808 08:25:03.472892 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 08:25:03.473055 27057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:03.469128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.194:43833 (local address 127.0.0.1:51840)
0808 08:25:03.469352 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:03.469365 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:03.469474 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:03.469750 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:03.469753 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:03.469760 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 08:25:03.469864 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:03.469870 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.470513 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.470516 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:03.471462 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:03.471473 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.471886 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.471890 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:03.471990 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:03.472044 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 08:25:03.472712 (+ 668us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 08:25:03.472728 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 08:25:03.472833 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 08:25:03.472918 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127}
I20260808 08:25:03.478519 26907 raft_consensus.cc:1261] T 134f85d1da2f4127ad90c2a3803f6842 P e454ff6295c54d31877c4683dc9a639b [term 1 FOLLOWER]: Refusing update from remote peer f49ac766475b4a0990668efe159a6f7b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:25:03.478554 26445 raft_consensus.cc:1261] T 134f85d1da2f4127ad90c2a3803f6842 P 4528d69c95104a80aef5b734b2bce43e [term 1 FOLLOWER]: Refusing update from remote peer f49ac766475b4a0990668efe159a6f7b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:25:03.479238 27039 consensus_queue.cc:1035] T 134f85d1da2f4127ad90c2a3803f6842 P f49ac766475b4a0990668efe159a6f7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:25:03.479403 27046 consensus_queue.cc:1035] T 134f85d1da2f4127ad90c2a3803f6842 P f49ac766475b4a0990668efe159a6f7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:25:03.483351 27040 tablet_replica.cc:439] T 134f85d1da2f4127ad90c2a3803f6842 P e454ff6295c54d31877c4683dc9a639b: TxnStatusTablet state changed. Reason: New leader f49ac766475b4a0990668efe159a6f7b. Latest consensus state: current_term: 1 leader_uuid: "f49ac766475b4a0990668efe159a6f7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } } }
I20260808 08:25:03.483470 27040 tablet_replica.cc:442] T 134f85d1da2f4127ad90c2a3803f6842 P e454ff6295c54d31877c4683dc9a639b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 08:25:03.484037 27041 tablet_replica.cc:439] T 134f85d1da2f4127ad90c2a3803f6842 P 4528d69c95104a80aef5b734b2bce43e: TxnStatusTablet state changed. Reason: New leader f49ac766475b4a0990668efe159a6f7b. Latest consensus state: current_term: 1 leader_uuid: "f49ac766475b4a0990668efe159a6f7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } } }
I20260808 08:25:03.484160 27041 tablet_replica.cc:442] T 134f85d1da2f4127ad90c2a3803f6842 P 4528d69c95104a80aef5b734b2bce43e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 08:25:03.487182 27041 tablet_replica.cc:439] T 134f85d1da2f4127ad90c2a3803f6842 P 4528d69c95104a80aef5b734b2bce43e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f49ac766475b4a0990668efe159a6f7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } } }
I20260808 08:25:03.487300 27041 tablet_replica.cc:442] T 134f85d1da2f4127ad90c2a3803f6842 P 4528d69c95104a80aef5b734b2bce43e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 08:25:03.487460 27040 tablet_replica.cc:439] T 134f85d1da2f4127ad90c2a3803f6842 P e454ff6295c54d31877c4683dc9a639b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f49ac766475b4a0990668efe159a6f7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } } }
I20260808 08:25:03.487584 27040 tablet_replica.cc:442] T 134f85d1da2f4127ad90c2a3803f6842 P e454ff6295c54d31877c4683dc9a639b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 08:25:03.489243 27046 tablet_replica.cc:439] T 134f85d1da2f4127ad90c2a3803f6842 P f49ac766475b4a0990668efe159a6f7b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f49ac766475b4a0990668efe159a6f7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } } }
I20260808 08:25:03.489357 27046 tablet_replica.cc:442] T 134f85d1da2f4127ad90c2a3803f6842 P f49ac766475b4a0990668efe159a6f7b: This TxnStatusTablet replica's current role is: LEADER
I20260808 08:25:03.489526 27071 mvcc.cc:205] Tried to move back new op lower bound from 7316183054241456128 to 7316183054139469824. Current Snapshot: MvccSnapshot[applied={T|T < 7316183054241456128}]
I20260808 08:25:03.489871 27072 mvcc.cc:205] Tried to move back new op lower bound from 7316183054241456128 to 7316183054139469824. Current Snapshot: MvccSnapshot[applied={T|T < 7316183054241456128}]
I20260808 08:25:03.489908 27046 tablet_replica.cc:439] T 134f85d1da2f4127ad90c2a3803f6842 P f49ac766475b4a0990668efe159a6f7b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f49ac766475b4a0990668efe159a6f7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } } }
I20260808 08:25:03.490010 27046 tablet_replica.cc:442] T 134f85d1da2f4127ad90c2a3803f6842 P f49ac766475b4a0990668efe159a6f7b: This TxnStatusTablet replica's current role is: LEADER
I20260808 08:25:03.499017 27026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:03.496094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51842 (local address 127.22.235.194:43833)
0808 08:25:03.496265 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:03.496269 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:03.496282 (+ 13us) server_negotiation.cc:407] Connection header received
0808 08:25:03.496318 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:03.496320 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:03.496359 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:03.496418 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:25:03.496717 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.497219 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.498059 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.498266 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.498350 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:03.498609 (+ 259us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:25:03.498676 (+ 67us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 08:25:03.498734 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:03.498829 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 08:25:03.498878 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 08:25:03.509583 27026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:03.506289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:35285 (local address 127.22.235.194:43833)
0808 08:25:03.506392 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:03.506397 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:03.506578 (+ 181us) server_negotiation.cc:407] Connection header received
0808 08:25:03.506705 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:03.506708 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:03.506745 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:03.506830 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:25:03.507043 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.507613 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.508510 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.508798 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.508889 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:03.509189 (+ 300us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:25:03.509256 (+ 67us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 08:25:03.509312 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:03.509402 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 08:25:03.509440 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 08:25:03.509600 27085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:03.506250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.194:43833 (local address 127.22.235.194:35285)
0808 08:25:03.506505 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:03.506516 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:03.506611 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:03.506831 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:03.506834 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:03.506841 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 08:25:03.506950 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:03.506956 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.507701 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.507704 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:03.508410 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:03.508420 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.508890 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.508893 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:03.509006 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:03.509047 (+ 41us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 08:25:03.509324 (+ 277us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 08:25:03.509330 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 08:25:03.509388 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 08:25:03.509460 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":89,"threads_started":1}
I20260808 08:25:03.516909 26445 raft_consensus.cc:1261] T a6d62ca3ff214ef69a97fb2a4039c2f6 P 4528d69c95104a80aef5b734b2bce43e [term 1 FOLLOWER]: Refusing update from remote peer f49ac766475b4a0990668efe159a6f7b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:25:03.516974 26907 raft_consensus.cc:1261] T a6d62ca3ff214ef69a97fb2a4039c2f6 P e454ff6295c54d31877c4683dc9a639b [term 1 FOLLOWER]: Refusing update from remote peer f49ac766475b4a0990668efe159a6f7b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:25:03.517397 27075 consensus_queue.cc:1035] T a6d62ca3ff214ef69a97fb2a4039c2f6 P f49ac766475b4a0990668efe159a6f7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:25:03.517683 27046 consensus_queue.cc:1035] T a6d62ca3ff214ef69a97fb2a4039c2f6 P f49ac766475b4a0990668efe159a6f7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:25:03.520263 27069 mvcc.cc:205] Tried to move back new op lower bound from 7316183054402097152 to 7316183053503954944. Current Snapshot: MvccSnapshot[applied={T|T < 7316183054402097152}]
I20260808 08:25:03.557044 26195 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33236:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:25:03.557212 26195 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33236:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:25:03.559594 26195 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 14fef20eb9444ae995182431d0545751: Sending DeleteTablet for 3 replicas of tablet a6d62ca3ff214ef69a97fb2a4039c2f6
I20260808 08:25:03.560372 26872 tablet_service.cc:1514] Processing DeleteTablet for tablet a6d62ca3ff214ef69a97fb2a4039c2f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:25:03 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:48792
I20260808 08:25:03.560371 26415 tablet_service.cc:1514] Processing DeleteTablet for tablet a6d62ca3ff214ef69a97fb2a4039c2f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:25:03 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:48258
I20260808 08:25:03.561043 27091 tablet_replica.cc:330] T a6d62ca3ff214ef69a97fb2a4039c2f6 P 4528d69c95104a80aef5b734b2bce43e: stopping tablet replica
I20260808 08:25:03.561043 27092 tablet_replica.cc:330] T a6d62ca3ff214ef69a97fb2a4039c2f6 P e454ff6295c54d31877c4683dc9a639b: stopping tablet replica
I20260808 08:25:03.561056 26195 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33236:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 08:25:03.561378 27091 raft_consensus.cc:2229] T a6d62ca3ff214ef69a97fb2a4039c2f6 P 4528d69c95104a80aef5b734b2bce43e [term 1 FOLLOWER]: Raft consensus shutting down.
W20260808 08:25:03.561486 26195 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 08:25:03.561616 27091 raft_consensus.cc:2258] T a6d62ca3ff214ef69a97fb2a4039c2f6 P 4528d69c95104a80aef5b734b2bce43e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:25:03.561612 27092 raft_consensus.cc:2229] T a6d62ca3ff214ef69a97fb2a4039c2f6 P e454ff6295c54d31877c4683dc9a639b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 08:25:03.562003 27092 raft_consensus.cc:2258] T a6d62ca3ff214ef69a97fb2a4039c2f6 P e454ff6295c54d31877c4683dc9a639b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:25:03.562697 26643 tablet_service.cc:1514] Processing DeleteTablet for tablet a6d62ca3ff214ef69a97fb2a4039c2f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:25:03 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:51836
I20260808 08:25:03.563333 27092 ts_tablet_manager.cc:1905] T a6d62ca3ff214ef69a97fb2a4039c2f6 P e454ff6295c54d31877c4683dc9a639b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:25:03.563361 27093 tablet_replica.cc:330] T a6d62ca3ff214ef69a97fb2a4039c2f6 P f49ac766475b4a0990668efe159a6f7b: stopping tablet replica
I20260808 08:25:03.563640 27093 raft_consensus.cc:2229] T a6d62ca3ff214ef69a97fb2a4039c2f6 P f49ac766475b4a0990668efe159a6f7b [term 1 LEADER]: Raft consensus shutting down.
I20260808 08:25:03.563966 27093 raft_consensus.cc:2258] T a6d62ca3ff214ef69a97fb2a4039c2f6 P f49ac766475b4a0990668efe159a6f7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:25:03.565240 27093 ts_tablet_manager.cc:1905] T a6d62ca3ff214ef69a97fb2a4039c2f6 P f49ac766475b4a0990668efe159a6f7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:25:03.566538 27092 ts_tablet_manager.cc:1918] T a6d62ca3ff214ef69a97fb2a4039c2f6 P e454ff6295c54d31877c4683dc9a639b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 08:25:03.566637 27092 log.cc:1196] T a6d62ca3ff214ef69a97fb2a4039c2f6 P e454ff6295c54d31877c4683dc9a639b: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-2/wal/wals/a6d62ca3ff214ef69a97fb2a4039c2f6
I20260808 08:25:03.567070 26872 tablet_service.cc:1467] Processing CreateTablet for tablet 9e0a9ab87ad6498aa952e31b30bf2d71 (DEFAULT_TABLE table=test-table [id=c55a4b8b3adf4e25ab643132444020de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:25:03.567077 26415 tablet_service.cc:1467] Processing CreateTablet for tablet 9e0a9ab87ad6498aa952e31b30bf2d71 (DEFAULT_TABLE table=test-table [id=c55a4b8b3adf4e25ab643132444020de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:25:03.567317 26415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e0a9ab87ad6498aa952e31b30bf2d71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:03.567325 26872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e0a9ab87ad6498aa952e31b30bf2d71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:03.568182 27093 ts_tablet_manager.cc:1918] T a6d62ca3ff214ef69a97fb2a4039c2f6 P f49ac766475b4a0990668efe159a6f7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 08:25:03.568265 27093 log.cc:1196] T a6d62ca3ff214ef69a97fb2a4039c2f6 P f49ac766475b4a0990668efe159a6f7b: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-1/wal/wals/a6d62ca3ff214ef69a97fb2a4039c2f6
I20260808 08:25:03.568583 27093 ts_tablet_manager.cc:1939] T a6d62ca3ff214ef69a97fb2a4039c2f6 P f49ac766475b4a0990668efe159a6f7b: Deleting consensus metadata
I20260808 08:25:03.569476 27034 tablet_bootstrap.cc:492] T 9e0a9ab87ad6498aa952e31b30bf2d71 P e454ff6295c54d31877c4683dc9a639b: Bootstrap starting.
I20260808 08:25:03.569772 26172 catalog_manager.cc:4841] TS f49ac766475b4a0990668efe159a6f7b (127.22.235.194:43833): tablet a6d62ca3ff214ef69a97fb2a4039c2f6 (table test-table [id=afc4eb2ef695474b9f8831a1acfc6220]) successfully deleted
I20260808 08:25:03.569811 26643 tablet_service.cc:1467] Processing CreateTablet for tablet 9e0a9ab87ad6498aa952e31b30bf2d71 (DEFAULT_TABLE table=test-table [id=c55a4b8b3adf4e25ab643132444020de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:25:03.570053 26643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e0a9ab87ad6498aa952e31b30bf2d71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:03.570367 27034 tablet_bootstrap.cc:654] T 9e0a9ab87ad6498aa952e31b30bf2d71 P e454ff6295c54d31877c4683dc9a639b: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:03.571288 27092 ts_tablet_manager.cc:1939] T a6d62ca3ff214ef69a97fb2a4039c2f6 P e454ff6295c54d31877c4683dc9a639b: Deleting consensus metadata
I20260808 08:25:03.572207 27027 tablet_bootstrap.cc:492] T 9e0a9ab87ad6498aa952e31b30bf2d71 P f49ac766475b4a0990668efe159a6f7b: Bootstrap starting.
I20260808 08:25:03.573115 27027 tablet_bootstrap.cc:654] T 9e0a9ab87ad6498aa952e31b30bf2d71 P f49ac766475b4a0990668efe159a6f7b: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:03.573824 26172 catalog_manager.cc:4841] TS e454ff6295c54d31877c4683dc9a639b (127.22.235.195:44217): tablet a6d62ca3ff214ef69a97fb2a4039c2f6 (table test-table [id=afc4eb2ef695474b9f8831a1acfc6220]) successfully deleted
I20260808 08:25:03.574661 27027 tablet_bootstrap.cc:492] T 9e0a9ab87ad6498aa952e31b30bf2d71 P f49ac766475b4a0990668efe159a6f7b: No bootstrap required, opened a new log
I20260808 08:25:03.574735 27027 ts_tablet_manager.cc:1397] T 9e0a9ab87ad6498aa952e31b30bf2d71 P f49ac766475b4a0990668efe159a6f7b: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260808 08:25:03.575143 27027 raft_consensus.cc:348] T 9e0a9ab87ad6498aa952e31b30bf2d71 P f49ac766475b4a0990668efe159a6f7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } }
I20260808 08:25:03.575246 27027 raft_consensus.cc:374] T 9e0a9ab87ad6498aa952e31b30bf2d71 P f49ac766475b4a0990668efe159a6f7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:03.575302 27027 raft_consensus.cc:729] T 9e0a9ab87ad6498aa952e31b30bf2d71 P f49ac766475b4a0990668efe159a6f7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f49ac766475b4a0990668efe159a6f7b, State: Initialized, Role: FOLLOWER
I20260808 08:25:03.575412 27027 consensus_queue.cc:260] T 9e0a9ab87ad6498aa952e31b30bf2d71 P f49ac766475b4a0990668efe159a6f7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } }
I20260808 08:25:03.575639 27027 ts_tablet_manager.cc:1428] T 9e0a9ab87ad6498aa952e31b30bf2d71 P f49ac766475b4a0990668efe159a6f7b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 08:25:03.576943 27030 tablet_bootstrap.cc:492] T 9e0a9ab87ad6498aa952e31b30bf2d71 P 4528d69c95104a80aef5b734b2bce43e: Bootstrap starting.
I20260808 08:25:03.577692 27034 tablet_bootstrap.cc:492] T 9e0a9ab87ad6498aa952e31b30bf2d71 P e454ff6295c54d31877c4683dc9a639b: No bootstrap required, opened a new log
I20260808 08:25:03.577773 27034 ts_tablet_manager.cc:1397] T 9e0a9ab87ad6498aa952e31b30bf2d71 P e454ff6295c54d31877c4683dc9a639b: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260808 08:25:03.577901 27030 tablet_bootstrap.cc:654] T 9e0a9ab87ad6498aa952e31b30bf2d71 P 4528d69c95104a80aef5b734b2bce43e: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:03.578233 27034 raft_consensus.cc:348] T 9e0a9ab87ad6498aa952e31b30bf2d71 P e454ff6295c54d31877c4683dc9a639b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } }
I20260808 08:25:03.578339 27034 raft_consensus.cc:374] T 9e0a9ab87ad6498aa952e31b30bf2d71 P e454ff6295c54d31877c4683dc9a639b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:03.578370 27034 raft_consensus.cc:729] T 9e0a9ab87ad6498aa952e31b30bf2d71 P e454ff6295c54d31877c4683dc9a639b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e454ff6295c54d31877c4683dc9a639b, State: Initialized, Role: FOLLOWER
I20260808 08:25:03.578495 27034 consensus_queue.cc:260] T 9e0a9ab87ad6498aa952e31b30bf2d71 P e454ff6295c54d31877c4683dc9a639b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } }
I20260808 08:25:03.578732 27034 ts_tablet_manager.cc:1428] T 9e0a9ab87ad6498aa952e31b30bf2d71 P e454ff6295c54d31877c4683dc9a639b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 08:25:03.579088 27030 tablet_bootstrap.cc:492] T 9e0a9ab87ad6498aa952e31b30bf2d71 P 4528d69c95104a80aef5b734b2bce43e: No bootstrap required, opened a new log
I20260808 08:25:03.579177 27030 ts_tablet_manager.cc:1397] T 9e0a9ab87ad6498aa952e31b30bf2d71 P 4528d69c95104a80aef5b734b2bce43e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 08:25:03.579669 27030 raft_consensus.cc:348] T 9e0a9ab87ad6498aa952e31b30bf2d71 P 4528d69c95104a80aef5b734b2bce43e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } }
I20260808 08:25:03.579784 27030 raft_consensus.cc:374] T 9e0a9ab87ad6498aa952e31b30bf2d71 P 4528d69c95104a80aef5b734b2bce43e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:03.579818 27030 raft_consensus.cc:729] T 9e0a9ab87ad6498aa952e31b30bf2d71 P 4528d69c95104a80aef5b734b2bce43e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4528d69c95104a80aef5b734b2bce43e, State: Initialized, Role: FOLLOWER
I20260808 08:25:03.579936 27030 consensus_queue.cc:260] T 9e0a9ab87ad6498aa952e31b30bf2d71 P 4528d69c95104a80aef5b734b2bce43e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } }
I20260808 08:25:03.580181 27030 ts_tablet_manager.cc:1428] T 9e0a9ab87ad6498aa952e31b30bf2d71 P 4528d69c95104a80aef5b734b2bce43e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 08:25:03.585718 27091 ts_tablet_manager.cc:1905] T a6d62ca3ff214ef69a97fb2a4039c2f6 P 4528d69c95104a80aef5b734b2bce43e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:25:03.588853 27091 ts_tablet_manager.cc:1918] T a6d62ca3ff214ef69a97fb2a4039c2f6 P 4528d69c95104a80aef5b734b2bce43e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 08:25:03.588976 27091 log.cc:1196] T a6d62ca3ff214ef69a97fb2a4039c2f6 P 4528d69c95104a80aef5b734b2bce43e: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-0/wal/wals/a6d62ca3ff214ef69a97fb2a4039c2f6
I20260808 08:25:03.589339 27091 ts_tablet_manager.cc:1939] T a6d62ca3ff214ef69a97fb2a4039c2f6 P 4528d69c95104a80aef5b734b2bce43e: Deleting consensus metadata
I20260808 08:25:03.590727 26170 catalog_manager.cc:4841] TS 4528d69c95104a80aef5b734b2bce43e (127.22.235.193:41383): tablet a6d62ca3ff214ef69a97fb2a4039c2f6 (table test-table [id=afc4eb2ef695474b9f8831a1acfc6220]) successfully deleted
I20260808 08:25:03.711445 27040 raft_consensus.cc:482] T 9e0a9ab87ad6498aa952e31b30bf2d71 P e454ff6295c54d31877c4683dc9a639b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 08:25:03.711665 27040 raft_consensus.cc:504] T 9e0a9ab87ad6498aa952e31b30bf2d71 P e454ff6295c54d31877c4683dc9a639b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } }
I20260808 08:25:03.712908 27040 leader_election.cc:288] T 9e0a9ab87ad6498aa952e31b30bf2d71 P e454ff6295c54d31877c4683dc9a639b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4528d69c95104a80aef5b734b2bce43e (127.22.235.193:41383), f49ac766475b4a0990668efe159a6f7b (127.22.235.194:43833)
I20260808 08:25:03.717396 27026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:03.713568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:42373 (local address 127.22.235.194:43833)
0808 08:25:03.713695 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:03.713699 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:03.713715 (+ 16us) server_negotiation.cc:407] Connection header received
0808 08:25:03.713772 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:03.713775 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:03.713825 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:03.713915 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:25:03.714200 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.714961 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.716368 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.716828 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.716945 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:03.717004 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:03.717183 (+ 179us) server_negotiation.cc:299] Negotiation successful
0808 08:25:03.717264 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 08:25:03.717540 27022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:03.713333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:40755 (local address 127.22.235.193:41383)
0808 08:25:03.713464 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:03.713469 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:03.713595 (+ 126us) server_negotiation.cc:407] Connection header received
0808 08:25:03.713815 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:03.713819 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:03.713863 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:03.713908 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:25:03.714292 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.714775 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.716349 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.716989 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.717112 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:03.717164 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:03.717326 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 08:25:03.717401 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 08:25:03.717772 27101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:03.713175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.194:43833 (local address 127.22.235.195:42373)
0808 08:25:03.713570 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:03.713581 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:03.713689 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:03.713947 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:03.713950 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:03.713969 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:25:03.714084 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:03.714092 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.715077 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.715080 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:03.716206 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:03.716217 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.716941 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.716945 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:03.717040 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:03.717066 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 08:25:03.717561 (+ 495us) client_negotiation.cc:240] Negotiation successful
0808 08:25:03.717627 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"mutex_wait_us":24,"spinlock_wait_cycles":21632,"thread_start_us":64,"threads_started":1}
I20260808 08:25:03.717774 27099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:03.713148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:41383 (local address 127.22.235.195:40755)
0808 08:25:03.713496 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:03.713512 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:03.713690 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:03.714043 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:03.714045 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:03.714058 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:25:03.714180 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:03.714187 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.714883 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.714886 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:03.716206 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:03.716217 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.717124 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.717128 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:03.717210 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:03.717230 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 08:25:03.717294 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 08:25:03.717359 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":45,"spinlock_wait_cycles":12288,"thread_start_us":87,"threads_started":1}
I20260808 08:25:03.718250 26445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e0a9ab87ad6498aa952e31b30bf2d71" candidate_uuid: "e454ff6295c54d31877c4683dc9a639b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4528d69c95104a80aef5b734b2bce43e" is_pre_election: true
I20260808 08:25:03.718405 26445 raft_consensus.cc:2454] T 9e0a9ab87ad6498aa952e31b30bf2d71 P 4528d69c95104a80aef5b734b2bce43e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e454ff6295c54d31877c4683dc9a639b in term 0.
I20260808 08:25:03.718532 26680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e0a9ab87ad6498aa952e31b30bf2d71" candidate_uuid: "e454ff6295c54d31877c4683dc9a639b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f49ac766475b4a0990668efe159a6f7b" is_pre_election: true
I20260808 08:25:03.718760 26680 raft_consensus.cc:2454] T 9e0a9ab87ad6498aa952e31b30bf2d71 P f49ac766475b4a0990668efe159a6f7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e454ff6295c54d31877c4683dc9a639b in term 0.
I20260808 08:25:03.718766 26796 leader_election.cc:302] T 9e0a9ab87ad6498aa952e31b30bf2d71 P e454ff6295c54d31877c4683dc9a639b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4528d69c95104a80aef5b734b2bce43e, e454ff6295c54d31877c4683dc9a639b; no voters:
I20260808 08:25:03.719022 27040 raft_consensus.cc:2781] T 9e0a9ab87ad6498aa952e31b30bf2d71 P e454ff6295c54d31877c4683dc9a639b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 08:25:03.719081 27040 raft_consensus.cc:482] T 9e0a9ab87ad6498aa952e31b30bf2d71 P e454ff6295c54d31877c4683dc9a639b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 08:25:03.719120 27040 raft_consensus.cc:3037] T 9e0a9ab87ad6498aa952e31b30bf2d71 P e454ff6295c54d31877c4683dc9a639b [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:03.720170 27040 raft_consensus.cc:504] T 9e0a9ab87ad6498aa952e31b30bf2d71 P e454ff6295c54d31877c4683dc9a639b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } }
I20260808 08:25:03.720510 27040 leader_election.cc:288] T 9e0a9ab87ad6498aa952e31b30bf2d71 P e454ff6295c54d31877c4683dc9a639b [CANDIDATE]: Term 1 election: Requested vote from peers 4528d69c95104a80aef5b734b2bce43e (127.22.235.193:41383), f49ac766475b4a0990668efe159a6f7b (127.22.235.194:43833)
I20260808 08:25:03.720896 26680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e0a9ab87ad6498aa952e31b30bf2d71" candidate_uuid: "e454ff6295c54d31877c4683dc9a639b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f49ac766475b4a0990668efe159a6f7b"
I20260808 08:25:03.720896 26445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e0a9ab87ad6498aa952e31b30bf2d71" candidate_uuid: "e454ff6295c54d31877c4683dc9a639b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4528d69c95104a80aef5b734b2bce43e"
I20260808 08:25:03.721016 26445 raft_consensus.cc:3037] T 9e0a9ab87ad6498aa952e31b30bf2d71 P 4528d69c95104a80aef5b734b2bce43e [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:03.721016 26680 raft_consensus.cc:3037] T 9e0a9ab87ad6498aa952e31b30bf2d71 P f49ac766475b4a0990668efe159a6f7b [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:03.722093 26680 raft_consensus.cc:2454] T 9e0a9ab87ad6498aa952e31b30bf2d71 P f49ac766475b4a0990668efe159a6f7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e454ff6295c54d31877c4683dc9a639b in term 1.
I20260808 08:25:03.722093 26445 raft_consensus.cc:2454] T 9e0a9ab87ad6498aa952e31b30bf2d71 P 4528d69c95104a80aef5b734b2bce43e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e454ff6295c54d31877c4683dc9a639b in term 1.
I20260808 08:25:03.722381 26800 leader_election.cc:302] T 9e0a9ab87ad6498aa952e31b30bf2d71 P e454ff6295c54d31877c4683dc9a639b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e454ff6295c54d31877c4683dc9a639b, f49ac766475b4a0990668efe159a6f7b; no voters:
I20260808 08:25:03.722656 27040 raft_consensus.cc:2781] T 9e0a9ab87ad6498aa952e31b30bf2d71 P e454ff6295c54d31877c4683dc9a639b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:25:03.722887 27040 raft_consensus.cc:686] T 9e0a9ab87ad6498aa952e31b30bf2d71 P e454ff6295c54d31877c4683dc9a639b [term 1 LEADER]: Becoming Leader. State: Replica: e454ff6295c54d31877c4683dc9a639b, State: Running, Role: LEADER
I20260808 08:25:03.723150 27040 consensus_queue.cc:237] T 9e0a9ab87ad6498aa952e31b30bf2d71 P e454ff6295c54d31877c4683dc9a639b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } }
I20260808 08:25:03.725718 26195 catalog_manager.cc:5495] T 9e0a9ab87ad6498aa952e31b30bf2d71 P e454ff6295c54d31877c4683dc9a639b reported cstate change: term changed from 0 to 1, leader changed from <none> to e454ff6295c54d31877c4683dc9a639b (127.22.235.195). New cstate: current_term: 1 leader_uuid: "e454ff6295c54d31877c4683dc9a639b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } health_report { overall_health: HEALTHY } } }
I20260808 08:25:03.744521 27021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:03.740989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48794 (local address 127.22.235.195:44217)
0808 08:25:03.741094 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:03.741098 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:03.741139 (+ 41us) server_negotiation.cc:407] Connection header received
0808 08:25:03.741322 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:03.741324 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:03.741382 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:03.741485 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:25:03.741698 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.742259 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.743058 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.743316 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.743405 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:03.743666 (+ 261us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:25:03.744129 (+ 463us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 08:25:03.744233 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:03.744327 (+ 94us) server_negotiation.cc:299] Negotiation successful
0808 08:25:03.744380 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 08:25:03.755327 27026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:03.751861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:59545 (local address 127.22.235.194:43833)
0808 08:25:03.751992 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:03.751997 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:03.752135 (+ 138us) server_negotiation.cc:407] Connection header received
0808 08:25:03.752226 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:03.752229 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:03.752272 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:03.752333 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:25:03.752571 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.753106 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.754302 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.754534 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.754617 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:03.754899 (+ 282us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:25:03.755001 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 08:25:03.755062 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:03.755146 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 08:25:03.755190 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 08:25:03.755353 27107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:03.751729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.194:43833 (local address 127.22.235.195:59545)
0808 08:25:03.752059 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:03.752072 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:03.752160 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:03.752343 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:03.752345 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:03.752351 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 08:25:03.752461 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:03.752490 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.753183 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.753186 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:03.754200 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:03.754211 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.754641 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.754645 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:03.754732 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:03.754775 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 08:25:03.755070 (+ 295us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 08:25:03.755077 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 08:25:03.755138 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 08:25:03.755215 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":102,"threads_started":1}
I20260808 08:25:03.761137 26445 raft_consensus.cc:1261] T 9e0a9ab87ad6498aa952e31b30bf2d71 P 4528d69c95104a80aef5b734b2bce43e [term 1 FOLLOWER]: Refusing update from remote peer e454ff6295c54d31877c4683dc9a639b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:25:03.761334 26680 raft_consensus.cc:1261] T 9e0a9ab87ad6498aa952e31b30bf2d71 P f49ac766475b4a0990668efe159a6f7b [term 1 FOLLOWER]: Refusing update from remote peer e454ff6295c54d31877c4683dc9a639b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:25:03.761616 27040 consensus_queue.cc:1035] T 9e0a9ab87ad6498aa952e31b30bf2d71 P e454ff6295c54d31877c4683dc9a639b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:25:03.761843 27103 consensus_queue.cc:1035] T 9e0a9ab87ad6498aa952e31b30bf2d71 P e454ff6295c54d31877c4683dc9a639b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:25:03.778821 27021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:03.775636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:37613 (local address 127.22.235.195:44217)
0808 08:25:03.775738 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:03.775743 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:03.775755 (+ 12us) server_negotiation.cc:407] Connection header received
0808 08:25:03.775908 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:03.775911 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:03.775964 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:03.776045 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:25:03.776271 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.776873 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.777803 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.778104 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.778184 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:03.778423 (+ 239us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:25:03.778500 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 08:25:03.778557 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:03.778646 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 08:25:03.778700 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 08:25:03.778852 27085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:03.775532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.195:44217 (local address 127.22.235.194:37613)
0808 08:25:03.775707 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:03.775721 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:03.775814 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:03.776060 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:03.776062 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:03.776071 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 08:25:03.776174 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:03.776180 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.776967 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.776969 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:03.777676 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:03.777687 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:03.778206 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:03.778210 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:03.778287 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:03.778315 (+ 28us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 08:25:03.778565 (+ 250us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 08:25:03.778572 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 08:25:03.778637 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 08:25:03.778725 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
W20260808 08:25:03.779505 27071 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 08:25:05.339085 26195 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33236:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:25:05.339232 26195 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33236:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:25:05.341509 26195 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 14fef20eb9444ae995182431d0545751: Sending DeleteTablet for 3 replicas of tablet 9e0a9ab87ad6498aa952e31b30bf2d71
I20260808 08:25:05.342027 26415 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e0a9ab87ad6498aa952e31b30bf2d71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:25:05 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:48258
I20260808 08:25:05.342119 26643 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e0a9ab87ad6498aa952e31b30bf2d71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:25:05 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:51836
I20260808 08:25:05.342163 26872 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e0a9ab87ad6498aa952e31b30bf2d71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:25:05 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:48792
I20260808 08:25:05.342444 27170 tablet_replica.cc:330] T 9e0a9ab87ad6498aa952e31b30bf2d71 P f49ac766475b4a0990668efe159a6f7b: stopping tablet replica
I20260808 08:25:05.342572 27170 raft_consensus.cc:2229] T 9e0a9ab87ad6498aa952e31b30bf2d71 P f49ac766475b4a0990668efe159a6f7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 08:25:05.342672 27170 raft_consensus.cc:2258] T 9e0a9ab87ad6498aa952e31b30bf2d71 P f49ac766475b4a0990668efe159a6f7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:25:05.342888 27171 tablet_replica.cc:330] T 9e0a9ab87ad6498aa952e31b30bf2d71 P e454ff6295c54d31877c4683dc9a639b: stopping tablet replica
I20260808 08:25:05.343003 27171 raft_consensus.cc:2229] T 9e0a9ab87ad6498aa952e31b30bf2d71 P e454ff6295c54d31877c4683dc9a639b [term 1 LEADER]: Raft consensus shutting down.
I20260808 08:25:05.343304 27171 raft_consensus.cc:2258] T 9e0a9ab87ad6498aa952e31b30bf2d71 P e454ff6295c54d31877c4683dc9a639b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:25:05.343822 27170 ts_tablet_manager.cc:1905] T 9e0a9ab87ad6498aa952e31b30bf2d71 P f49ac766475b4a0990668efe159a6f7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:25:05.346266 27170 ts_tablet_manager.cc:1918] T 9e0a9ab87ad6498aa952e31b30bf2d71 P f49ac766475b4a0990668efe159a6f7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 08:25:05.346334 27170 log.cc:1196] T 9e0a9ab87ad6498aa952e31b30bf2d71 P f49ac766475b4a0990668efe159a6f7b: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-1/wal/wals/9e0a9ab87ad6498aa952e31b30bf2d71
I20260808 08:25:05.346545 27170 ts_tablet_manager.cc:1939] T 9e0a9ab87ad6498aa952e31b30bf2d71 P f49ac766475b4a0990668efe159a6f7b: Deleting consensus metadata
I20260808 08:25:05.347159 26172 catalog_manager.cc:4841] TS f49ac766475b4a0990668efe159a6f7b (127.22.235.194:43833): tablet 9e0a9ab87ad6498aa952e31b30bf2d71 (table test-table [id=c55a4b8b3adf4e25ab643132444020de]) successfully deleted
I20260808 08:25:05.347388 27171 ts_tablet_manager.cc:1905] T 9e0a9ab87ad6498aa952e31b30bf2d71 P e454ff6295c54d31877c4683dc9a639b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:25:05.348804 27169 tablet_replica.cc:330] T 9e0a9ab87ad6498aa952e31b30bf2d71 P 4528d69c95104a80aef5b734b2bce43e: stopping tablet replica
I20260808 08:25:05.348986 27169 raft_consensus.cc:2229] T 9e0a9ab87ad6498aa952e31b30bf2d71 P 4528d69c95104a80aef5b734b2bce43e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 08:25:05.349105 27169 raft_consensus.cc:2258] T 9e0a9ab87ad6498aa952e31b30bf2d71 P 4528d69c95104a80aef5b734b2bce43e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:25:05.349634 27169 ts_tablet_manager.cc:1905] T 9e0a9ab87ad6498aa952e31b30bf2d71 P 4528d69c95104a80aef5b734b2bce43e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:25:05.349723 27171 ts_tablet_manager.cc:1918] T 9e0a9ab87ad6498aa952e31b30bf2d71 P e454ff6295c54d31877c4683dc9a639b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 08:25:05.349802 27171 log.cc:1196] T 9e0a9ab87ad6498aa952e31b30bf2d71 P e454ff6295c54d31877c4683dc9a639b: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-2/wal/wals/9e0a9ab87ad6498aa952e31b30bf2d71
I20260808 08:25:05.350082 27171 ts_tablet_manager.cc:1939] T 9e0a9ab87ad6498aa952e31b30bf2d71 P e454ff6295c54d31877c4683dc9a639b: Deleting consensus metadata
I20260808 08:25:05.350725 26172 catalog_manager.cc:4841] TS e454ff6295c54d31877c4683dc9a639b (127.22.235.195:44217): tablet 9e0a9ab87ad6498aa952e31b30bf2d71 (table test-table [id=c55a4b8b3adf4e25ab643132444020de]) successfully deleted
I20260808 08:25:05.352319 27169 ts_tablet_manager.cc:1918] T 9e0a9ab87ad6498aa952e31b30bf2d71 P 4528d69c95104a80aef5b734b2bce43e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 08:25:05.352401 27169 log.cc:1196] T 9e0a9ab87ad6498aa952e31b30bf2d71 P 4528d69c95104a80aef5b734b2bce43e: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-0/wal/wals/9e0a9ab87ad6498aa952e31b30bf2d71
I20260808 08:25:05.352717 27169 ts_tablet_manager.cc:1939] T 9e0a9ab87ad6498aa952e31b30bf2d71 P 4528d69c95104a80aef5b734b2bce43e: Deleting consensus metadata
I20260808 08:25:05.353447 26170 catalog_manager.cc:4841] TS 4528d69c95104a80aef5b734b2bce43e (127.22.235.193:41383): tablet 9e0a9ab87ad6498aa952e31b30bf2d71 (table test-table [id=c55a4b8b3adf4e25ab643132444020de]) successfully deleted
Aug 08 08:25:05 dist-test-slave-f7th krb5kdc[26123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177505, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 08:25:05 dist-test-slave-f7th krb5kdc[26123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177505, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:25:05.374366 27181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:05.362028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33256 (local address 127.22.235.254:36447)
0808 08:25:05.362273 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:05.362281 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:05.362295 (+ 14us) server_negotiation.cc:407] Connection header received
0808 08:25:05.362439 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:05.362442 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:05.362485 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:05.362573 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:05.363239 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:05.363782 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:05.364577 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:05.364876 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:05.364967 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:05.371434 (+ 6467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:05.371438 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:05.371440 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:05.371467 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:05.372918 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:05.373317 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:05.373319 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:05.373320 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:05.373402 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:05.373664 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:05.373667 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:05.373668 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:05.373846 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:05.373949 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:05.374133 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 08:25:05.374234 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":85,"threads_started":1}
I20260808 08:25:05.376835 26195 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33256:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 08:25:05.377285 26195 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 08:25:05.381726 26415 tablet_service.cc:1467] Processing CreateTablet for tablet d3a1fa4023694195ac3afc03351cb40f (DEFAULT_TABLE table=test-table [id=7cce611c91d84eceb9a2894594b8035a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:25:05.381826 26643 tablet_service.cc:1467] Processing CreateTablet for tablet d3a1fa4023694195ac3afc03351cb40f (DEFAULT_TABLE table=test-table [id=7cce611c91d84eceb9a2894594b8035a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:25:05.381863 26872 tablet_service.cc:1467] Processing CreateTablet for tablet d3a1fa4023694195ac3afc03351cb40f (DEFAULT_TABLE table=test-table [id=7cce611c91d84eceb9a2894594b8035a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:25:05.382035 26415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3a1fa4023694195ac3afc03351cb40f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:05.382138 26872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3a1fa4023694195ac3afc03351cb40f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:05.382138 26643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3a1fa4023694195ac3afc03351cb40f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:05.384050 27182 tablet_bootstrap.cc:492] T d3a1fa4023694195ac3afc03351cb40f P 4528d69c95104a80aef5b734b2bce43e: Bootstrap starting.
I20260808 08:25:05.384552 27183 tablet_bootstrap.cc:492] T d3a1fa4023694195ac3afc03351cb40f P f49ac766475b4a0990668efe159a6f7b: Bootstrap starting.
I20260808 08:25:05.384953 27182 tablet_bootstrap.cc:654] T d3a1fa4023694195ac3afc03351cb40f P 4528d69c95104a80aef5b734b2bce43e: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:05.385421 27183 tablet_bootstrap.cc:654] T d3a1fa4023694195ac3afc03351cb40f P f49ac766475b4a0990668efe159a6f7b: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:05.386066 27185 tablet_bootstrap.cc:492] T d3a1fa4023694195ac3afc03351cb40f P e454ff6295c54d31877c4683dc9a639b: Bootstrap starting.
I20260808 08:25:05.386267 27182 tablet_bootstrap.cc:492] T d3a1fa4023694195ac3afc03351cb40f P 4528d69c95104a80aef5b734b2bce43e: No bootstrap required, opened a new log
I20260808 08:25:05.386353 27182 ts_tablet_manager.cc:1397] T d3a1fa4023694195ac3afc03351cb40f P 4528d69c95104a80aef5b734b2bce43e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 08:25:05.386802 27182 raft_consensus.cc:348] T d3a1fa4023694195ac3afc03351cb40f P 4528d69c95104a80aef5b734b2bce43e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } }
I20260808 08:25:05.386919 27185 tablet_bootstrap.cc:654] T d3a1fa4023694195ac3afc03351cb40f P e454ff6295c54d31877c4683dc9a639b: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:05.386924 27182 raft_consensus.cc:374] T d3a1fa4023694195ac3afc03351cb40f P 4528d69c95104a80aef5b734b2bce43e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:05.386957 27182 raft_consensus.cc:729] T d3a1fa4023694195ac3afc03351cb40f P 4528d69c95104a80aef5b734b2bce43e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4528d69c95104a80aef5b734b2bce43e, State: Initialized, Role: FOLLOWER
I20260808 08:25:05.387094 27182 consensus_queue.cc:260] T d3a1fa4023694195ac3afc03351cb40f P 4528d69c95104a80aef5b734b2bce43e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } }
I20260808 08:25:05.387465 27182 ts_tablet_manager.cc:1428] T d3a1fa4023694195ac3afc03351cb40f P 4528d69c95104a80aef5b734b2bce43e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 08:25:05.387799 27183 tablet_bootstrap.cc:492] T d3a1fa4023694195ac3afc03351cb40f P f49ac766475b4a0990668efe159a6f7b: No bootstrap required, opened a new log
I20260808 08:25:05.387877 27183 ts_tablet_manager.cc:1397] T d3a1fa4023694195ac3afc03351cb40f P f49ac766475b4a0990668efe159a6f7b: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 08:25:05.388281 27183 raft_consensus.cc:348] T d3a1fa4023694195ac3afc03351cb40f P f49ac766475b4a0990668efe159a6f7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } }
I20260808 08:25:05.388370 27183 raft_consensus.cc:374] T d3a1fa4023694195ac3afc03351cb40f P f49ac766475b4a0990668efe159a6f7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:05.388398 27183 raft_consensus.cc:729] T d3a1fa4023694195ac3afc03351cb40f P f49ac766475b4a0990668efe159a6f7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f49ac766475b4a0990668efe159a6f7b, State: Initialized, Role: FOLLOWER
I20260808 08:25:05.388530 27183 consensus_queue.cc:260] T d3a1fa4023694195ac3afc03351cb40f P f49ac766475b4a0990668efe159a6f7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } }
I20260808 08:25:05.388789 27183 ts_tablet_manager.cc:1428] T d3a1fa4023694195ac3afc03351cb40f P f49ac766475b4a0990668efe159a6f7b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 08:25:05.389364 27185 tablet_bootstrap.cc:492] T d3a1fa4023694195ac3afc03351cb40f P e454ff6295c54d31877c4683dc9a639b: No bootstrap required, opened a new log
I20260808 08:25:05.389447 27185 ts_tablet_manager.cc:1397] T d3a1fa4023694195ac3afc03351cb40f P e454ff6295c54d31877c4683dc9a639b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 08:25:05.389848 27185 raft_consensus.cc:348] T d3a1fa4023694195ac3afc03351cb40f P e454ff6295c54d31877c4683dc9a639b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } }
I20260808 08:25:05.389967 27185 raft_consensus.cc:374] T d3a1fa4023694195ac3afc03351cb40f P e454ff6295c54d31877c4683dc9a639b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:05.389999 27185 raft_consensus.cc:729] T d3a1fa4023694195ac3afc03351cb40f P e454ff6295c54d31877c4683dc9a639b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e454ff6295c54d31877c4683dc9a639b, State: Initialized, Role: FOLLOWER
I20260808 08:25:05.390189 27185 consensus_queue.cc:260] T d3a1fa4023694195ac3afc03351cb40f P e454ff6295c54d31877c4683dc9a639b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } }
I20260808 08:25:05.390460 27185 ts_tablet_manager.cc:1428] T d3a1fa4023694195ac3afc03351cb40f P e454ff6295c54d31877c4683dc9a639b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 08:25:05.622704 27154 raft_consensus.cc:482] T d3a1fa4023694195ac3afc03351cb40f P e454ff6295c54d31877c4683dc9a639b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 08:25:05.622855 27154 raft_consensus.cc:504] T d3a1fa4023694195ac3afc03351cb40f P e454ff6295c54d31877c4683dc9a639b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } }
I20260808 08:25:05.623281 27154 leader_election.cc:288] T d3a1fa4023694195ac3afc03351cb40f P e454ff6295c54d31877c4683dc9a639b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4528d69c95104a80aef5b734b2bce43e (127.22.235.193:41383), f49ac766475b4a0990668efe159a6f7b (127.22.235.194:43833)
I20260808 08:25:05.623745 26445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3a1fa4023694195ac3afc03351cb40f" candidate_uuid: "e454ff6295c54d31877c4683dc9a639b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4528d69c95104a80aef5b734b2bce43e" is_pre_election: true
I20260808 08:25:05.623744 26680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3a1fa4023694195ac3afc03351cb40f" candidate_uuid: "e454ff6295c54d31877c4683dc9a639b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f49ac766475b4a0990668efe159a6f7b" is_pre_election: true
I20260808 08:25:05.623896 26680 raft_consensus.cc:2454] T d3a1fa4023694195ac3afc03351cb40f P f49ac766475b4a0990668efe159a6f7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e454ff6295c54d31877c4683dc9a639b in term 0.
I20260808 08:25:05.623903 26445 raft_consensus.cc:2454] T d3a1fa4023694195ac3afc03351cb40f P 4528d69c95104a80aef5b734b2bce43e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e454ff6295c54d31877c4683dc9a639b in term 0.
I20260808 08:25:05.624257 26796 leader_election.cc:302] T d3a1fa4023694195ac3afc03351cb40f P e454ff6295c54d31877c4683dc9a639b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4528d69c95104a80aef5b734b2bce43e, e454ff6295c54d31877c4683dc9a639b, f49ac766475b4a0990668efe159a6f7b; no voters:
I20260808 08:25:05.624433 27154 raft_consensus.cc:2781] T d3a1fa4023694195ac3afc03351cb40f P e454ff6295c54d31877c4683dc9a639b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 08:25:05.624505 27154 raft_consensus.cc:482] T d3a1fa4023694195ac3afc03351cb40f P e454ff6295c54d31877c4683dc9a639b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 08:25:05.624542 27154 raft_consensus.cc:3037] T d3a1fa4023694195ac3afc03351cb40f P e454ff6295c54d31877c4683dc9a639b [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:05.625568 27154 raft_consensus.cc:504] T d3a1fa4023694195ac3afc03351cb40f P e454ff6295c54d31877c4683dc9a639b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } }
I20260808 08:25:05.625908 27154 leader_election.cc:288] T d3a1fa4023694195ac3afc03351cb40f P e454ff6295c54d31877c4683dc9a639b [CANDIDATE]: Term 1 election: Requested vote from peers 4528d69c95104a80aef5b734b2bce43e (127.22.235.193:41383), f49ac766475b4a0990668efe159a6f7b (127.22.235.194:43833)
I20260808 08:25:05.626294 26445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3a1fa4023694195ac3afc03351cb40f" candidate_uuid: "e454ff6295c54d31877c4683dc9a639b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4528d69c95104a80aef5b734b2bce43e"
I20260808 08:25:05.626315 26680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3a1fa4023694195ac3afc03351cb40f" candidate_uuid: "e454ff6295c54d31877c4683dc9a639b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f49ac766475b4a0990668efe159a6f7b"
I20260808 08:25:05.626415 26445 raft_consensus.cc:3037] T d3a1fa4023694195ac3afc03351cb40f P 4528d69c95104a80aef5b734b2bce43e [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:05.626431 26680 raft_consensus.cc:3037] T d3a1fa4023694195ac3afc03351cb40f P f49ac766475b4a0990668efe159a6f7b [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:05.627391 26445 raft_consensus.cc:2454] T d3a1fa4023694195ac3afc03351cb40f P 4528d69c95104a80aef5b734b2bce43e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e454ff6295c54d31877c4683dc9a639b in term 1.
I20260808 08:25:05.627398 26680 raft_consensus.cc:2454] T d3a1fa4023694195ac3afc03351cb40f P f49ac766475b4a0990668efe159a6f7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e454ff6295c54d31877c4683dc9a639b in term 1.
I20260808 08:25:05.627699 26800 leader_election.cc:302] T d3a1fa4023694195ac3afc03351cb40f P e454ff6295c54d31877c4683dc9a639b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e454ff6295c54d31877c4683dc9a639b, f49ac766475b4a0990668efe159a6f7b; no voters:
I20260808 08:25:05.627867 27154 raft_consensus.cc:2781] T d3a1fa4023694195ac3afc03351cb40f P e454ff6295c54d31877c4683dc9a639b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:25:05.627964 27154 raft_consensus.cc:686] T d3a1fa4023694195ac3afc03351cb40f P e454ff6295c54d31877c4683dc9a639b [term 1 LEADER]: Becoming Leader. State: Replica: e454ff6295c54d31877c4683dc9a639b, State: Running, Role: LEADER
I20260808 08:25:05.628108 27154 consensus_queue.cc:237] T d3a1fa4023694195ac3afc03351cb40f P e454ff6295c54d31877c4683dc9a639b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } }
I20260808 08:25:05.629896 26194 catalog_manager.cc:5495] T d3a1fa4023694195ac3afc03351cb40f P e454ff6295c54d31877c4683dc9a639b reported cstate change: term changed from 0 to 1, leader changed from <none> to e454ff6295c54d31877c4683dc9a639b (127.22.235.195). New cstate: current_term: 1 leader_uuid: "e454ff6295c54d31877c4683dc9a639b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } health_report { overall_health: HEALTHY } } }
I20260808 08:25:05.686314 27200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:05.683427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48806 (local address 127.22.235.195:44217)
0808 08:25:05.683689 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:05.683696 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:05.683708 (+ 12us) server_negotiation.cc:407] Connection header received
0808 08:25:05.683746 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:05.683748 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:05.683788 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:05.683855 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:25:05.684085 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:05.684600 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:05.685430 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:05.685667 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:05.685744 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:05.685938 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:25:05.686009 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 08:25:05.686058 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:05.686128 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 08:25:05.686170 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":106,"threads_started":1}
I20260808 08:25:05.692338 26445 raft_consensus.cc:1261] T d3a1fa4023694195ac3afc03351cb40f P 4528d69c95104a80aef5b734b2bce43e [term 1 FOLLOWER]: Refusing update from remote peer e454ff6295c54d31877c4683dc9a639b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:25:05.692674 26680 raft_consensus.cc:1261] T d3a1fa4023694195ac3afc03351cb40f P f49ac766475b4a0990668efe159a6f7b [term 1 FOLLOWER]: Refusing update from remote peer e454ff6295c54d31877c4683dc9a639b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:25:05.692773 27113 consensus_queue.cc:1035] T d3a1fa4023694195ac3afc03351cb40f P e454ff6295c54d31877c4683dc9a639b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:25:05.693121 27154 consensus_queue.cc:1035] T d3a1fa4023694195ac3afc03351cb40f P e454ff6295c54d31877c4683dc9a639b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:25:05.723124 26192 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:25:05.723268 26192 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:25:05.725090 26192 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 14fef20eb9444ae995182431d0545751: Sending DeleteTablet for 3 replicas of tablet d3a1fa4023694195ac3afc03351cb40f
I20260808 08:25:05.725613 26415 tablet_service.cc:1514] Processing DeleteTablet for tablet d3a1fa4023694195ac3afc03351cb40f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:25:05 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:48258
I20260808 08:25:05.725680 26872 tablet_service.cc:1514] Processing DeleteTablet for tablet d3a1fa4023694195ac3afc03351cb40f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:25:05 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:48792
I20260808 08:25:05.725805 27169 tablet_replica.cc:330] T d3a1fa4023694195ac3afc03351cb40f P 4528d69c95104a80aef5b734b2bce43e: stopping tablet replica
I20260808 08:25:05.725781 26643 tablet_service.cc:1514] Processing DeleteTablet for tablet d3a1fa4023694195ac3afc03351cb40f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:25:05 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:51836
I20260808 08:25:05.725852 27171 tablet_replica.cc:330] T d3a1fa4023694195ac3afc03351cb40f P e454ff6295c54d31877c4683dc9a639b: stopping tablet replica
I20260808 08:25:05.725911 27169 raft_consensus.cc:2229] T d3a1fa4023694195ac3afc03351cb40f P 4528d69c95104a80aef5b734b2bce43e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 08:25:05.725934 27170 tablet_replica.cc:330] T d3a1fa4023694195ac3afc03351cb40f P f49ac766475b4a0990668efe159a6f7b: stopping tablet replica
I20260808 08:25:05.725956 27171 raft_consensus.cc:2229] T d3a1fa4023694195ac3afc03351cb40f P e454ff6295c54d31877c4683dc9a639b [term 1 LEADER]: Raft consensus shutting down.
I20260808 08:25:05.726027 27170 raft_consensus.cc:2229] T d3a1fa4023694195ac3afc03351cb40f P f49ac766475b4a0990668efe159a6f7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 08:25:05.726035 27169 raft_consensus.cc:2258] T d3a1fa4023694195ac3afc03351cb40f P 4528d69c95104a80aef5b734b2bce43e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:25:05.726118 27170 raft_consensus.cc:2258] T d3a1fa4023694195ac3afc03351cb40f P f49ac766475b4a0990668efe159a6f7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:25:05.726141 27171 raft_consensus.cc:2258] T d3a1fa4023694195ac3afc03351cb40f P e454ff6295c54d31877c4683dc9a639b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:25:05.726182 26192 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33256:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 08:25:05.726586 26192 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 08:25:05.726924 27170 ts_tablet_manager.cc:1905] T d3a1fa4023694195ac3afc03351cb40f P f49ac766475b4a0990668efe159a6f7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:25:05.726955 27171 ts_tablet_manager.cc:1905] T d3a1fa4023694195ac3afc03351cb40f P e454ff6295c54d31877c4683dc9a639b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:25:05.729382 27170 ts_tablet_manager.cc:1918] T d3a1fa4023694195ac3afc03351cb40f P f49ac766475b4a0990668efe159a6f7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 08:25:05.729385 27171 ts_tablet_manager.cc:1918] T d3a1fa4023694195ac3afc03351cb40f P e454ff6295c54d31877c4683dc9a639b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 08:25:05.729465 27171 log.cc:1196] T d3a1fa4023694195ac3afc03351cb40f P e454ff6295c54d31877c4683dc9a639b: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-2/wal/wals/d3a1fa4023694195ac3afc03351cb40f
I20260808 08:25:05.729475 27170 log.cc:1196] T d3a1fa4023694195ac3afc03351cb40f P f49ac766475b4a0990668efe159a6f7b: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-1/wal/wals/d3a1fa4023694195ac3afc03351cb40f
I20260808 08:25:05.729722 27171 ts_tablet_manager.cc:1939] T d3a1fa4023694195ac3afc03351cb40f P e454ff6295c54d31877c4683dc9a639b: Deleting consensus metadata
I20260808 08:25:05.729735 27170 ts_tablet_manager.cc:1939] T d3a1fa4023694195ac3afc03351cb40f P f49ac766475b4a0990668efe159a6f7b: Deleting consensus metadata
I20260808 08:25:05.730439 26172 catalog_manager.cc:4841] TS f49ac766475b4a0990668efe159a6f7b (127.22.235.194:43833): tablet d3a1fa4023694195ac3afc03351cb40f (table test-table [id=7cce611c91d84eceb9a2894594b8035a]) successfully deleted
I20260808 08:25:05.730615 26172 catalog_manager.cc:4841] TS e454ff6295c54d31877c4683dc9a639b (127.22.235.195:44217): tablet d3a1fa4023694195ac3afc03351cb40f (table test-table [id=7cce611c91d84eceb9a2894594b8035a]) successfully deleted
I20260808 08:25:05.731218 27169 ts_tablet_manager.cc:1905] T d3a1fa4023694195ac3afc03351cb40f P 4528d69c95104a80aef5b734b2bce43e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:25:05.731417 26415 tablet_service.cc:1467] Processing CreateTablet for tablet 3080e977e9e94cbdba3c39b4ee8556ff (DEFAULT_TABLE table=test-table [id=24098858f8d94d5cad494a9aaf7b7ba3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:25:05.731688 26415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3080e977e9e94cbdba3c39b4ee8556ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:05.731676 26872 tablet_service.cc:1467] Processing CreateTablet for tablet 3080e977e9e94cbdba3c39b4ee8556ff (DEFAULT_TABLE table=test-table [id=24098858f8d94d5cad494a9aaf7b7ba3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:25:05.731688 26643 tablet_service.cc:1467] Processing CreateTablet for tablet 3080e977e9e94cbdba3c39b4ee8556ff (DEFAULT_TABLE table=test-table [id=24098858f8d94d5cad494a9aaf7b7ba3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:25:05.731891 26872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3080e977e9e94cbdba3c39b4ee8556ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:05.731901 26643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3080e977e9e94cbdba3c39b4ee8556ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:05.733680 27169 ts_tablet_manager.cc:1918] T d3a1fa4023694195ac3afc03351cb40f P 4528d69c95104a80aef5b734b2bce43e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 08:25:05.733748 27169 log.cc:1196] T d3a1fa4023694195ac3afc03351cb40f P 4528d69c95104a80aef5b734b2bce43e: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-0/wal/wals/d3a1fa4023694195ac3afc03351cb40f
I20260808 08:25:05.733964 27185 tablet_bootstrap.cc:492] T 3080e977e9e94cbdba3c39b4ee8556ff P e454ff6295c54d31877c4683dc9a639b: Bootstrap starting.
I20260808 08:25:05.734084 27182 tablet_bootstrap.cc:492] T 3080e977e9e94cbdba3c39b4ee8556ff P 4528d69c95104a80aef5b734b2bce43e: Bootstrap starting.
I20260808 08:25:05.734470 27183 tablet_bootstrap.cc:492] T 3080e977e9e94cbdba3c39b4ee8556ff P f49ac766475b4a0990668efe159a6f7b: Bootstrap starting.
I20260808 08:25:05.734754 27185 tablet_bootstrap.cc:654] T 3080e977e9e94cbdba3c39b4ee8556ff P e454ff6295c54d31877c4683dc9a639b: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:05.734917 27182 tablet_bootstrap.cc:654] T 3080e977e9e94cbdba3c39b4ee8556ff P 4528d69c95104a80aef5b734b2bce43e: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:05.735204 27183 tablet_bootstrap.cc:654] T 3080e977e9e94cbdba3c39b4ee8556ff P f49ac766475b4a0990668efe159a6f7b: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:05.735610 27169 ts_tablet_manager.cc:1939] T d3a1fa4023694195ac3afc03351cb40f P 4528d69c95104a80aef5b734b2bce43e: Deleting consensus metadata
I20260808 08:25:05.736171 26170 catalog_manager.cc:4841] TS 4528d69c95104a80aef5b734b2bce43e (127.22.235.193:41383): tablet d3a1fa4023694195ac3afc03351cb40f (table test-table [id=7cce611c91d84eceb9a2894594b8035a]) successfully deleted
I20260808 08:25:05.736397 27183 tablet_bootstrap.cc:492] T 3080e977e9e94cbdba3c39b4ee8556ff P f49ac766475b4a0990668efe159a6f7b: No bootstrap required, opened a new log
I20260808 08:25:05.736397 27185 tablet_bootstrap.cc:492] T 3080e977e9e94cbdba3c39b4ee8556ff P e454ff6295c54d31877c4683dc9a639b: No bootstrap required, opened a new log
I20260808 08:25:05.736483 27183 ts_tablet_manager.cc:1397] T 3080e977e9e94cbdba3c39b4ee8556ff P f49ac766475b4a0990668efe159a6f7b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 08:25:05.736483 27185 ts_tablet_manager.cc:1397] T 3080e977e9e94cbdba3c39b4ee8556ff P e454ff6295c54d31877c4683dc9a639b: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 08:25:05.736871 27183 raft_consensus.cc:348] T 3080e977e9e94cbdba3c39b4ee8556ff P f49ac766475b4a0990668efe159a6f7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } }
I20260808 08:25:05.736871 27185 raft_consensus.cc:348] T 3080e977e9e94cbdba3c39b4ee8556ff P e454ff6295c54d31877c4683dc9a639b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } }
I20260808 08:25:05.736985 27185 raft_consensus.cc:374] T 3080e977e9e94cbdba3c39b4ee8556ff P e454ff6295c54d31877c4683dc9a639b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:05.736999 27183 raft_consensus.cc:374] T 3080e977e9e94cbdba3c39b4ee8556ff P f49ac766475b4a0990668efe159a6f7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:05.737020 27185 raft_consensus.cc:729] T 3080e977e9e94cbdba3c39b4ee8556ff P e454ff6295c54d31877c4683dc9a639b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e454ff6295c54d31877c4683dc9a639b, State: Initialized, Role: FOLLOWER
I20260808 08:25:05.737040 27183 raft_consensus.cc:729] T 3080e977e9e94cbdba3c39b4ee8556ff P f49ac766475b4a0990668efe159a6f7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f49ac766475b4a0990668efe159a6f7b, State: Initialized, Role: FOLLOWER
I20260808 08:25:05.737140 27183 consensus_queue.cc:260] T 3080e977e9e94cbdba3c39b4ee8556ff P f49ac766475b4a0990668efe159a6f7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } }
I20260808 08:25:05.737140 27185 consensus_queue.cc:260] T 3080e977e9e94cbdba3c39b4ee8556ff P e454ff6295c54d31877c4683dc9a639b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } }
I20260808 08:25:05.737303 27182 tablet_bootstrap.cc:492] T 3080e977e9e94cbdba3c39b4ee8556ff P 4528d69c95104a80aef5b734b2bce43e: No bootstrap required, opened a new log
I20260808 08:25:05.737385 27182 ts_tablet_manager.cc:1397] T 3080e977e9e94cbdba3c39b4ee8556ff P 4528d69c95104a80aef5b734b2bce43e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 08:25:05.737396 27185 ts_tablet_manager.cc:1428] T 3080e977e9e94cbdba3c39b4ee8556ff P e454ff6295c54d31877c4683dc9a639b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 08:25:05.737396 27183 ts_tablet_manager.cc:1428] T 3080e977e9e94cbdba3c39b4ee8556ff P f49ac766475b4a0990668efe159a6f7b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 08:25:05.737756 27182 raft_consensus.cc:348] T 3080e977e9e94cbdba3c39b4ee8556ff P 4528d69c95104a80aef5b734b2bce43e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } }
I20260808 08:25:05.737866 27182 raft_consensus.cc:374] T 3080e977e9e94cbdba3c39b4ee8556ff P 4528d69c95104a80aef5b734b2bce43e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:05.737907 27182 raft_consensus.cc:729] T 3080e977e9e94cbdba3c39b4ee8556ff P 4528d69c95104a80aef5b734b2bce43e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4528d69c95104a80aef5b734b2bce43e, State: Initialized, Role: FOLLOWER
I20260808 08:25:05.738039 27182 consensus_queue.cc:260] T 3080e977e9e94cbdba3c39b4ee8556ff P 4528d69c95104a80aef5b734b2bce43e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } }
I20260808 08:25:05.738296 27182 ts_tablet_manager.cc:1428] T 3080e977e9e94cbdba3c39b4ee8556ff P 4528d69c95104a80aef5b734b2bce43e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 08:25:05.868918 27188 raft_consensus.cc:482] T 3080e977e9e94cbdba3c39b4ee8556ff P 4528d69c95104a80aef5b734b2bce43e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 08:25:05.869081 27188 raft_consensus.cc:504] T 3080e977e9e94cbdba3c39b4ee8556ff P 4528d69c95104a80aef5b734b2bce43e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } }
I20260808 08:25:05.870150 27188 leader_election.cc:288] T 3080e977e9e94cbdba3c39b4ee8556ff P 4528d69c95104a80aef5b734b2bce43e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f49ac766475b4a0990668efe159a6f7b (127.22.235.194:43833), e454ff6295c54d31877c4683dc9a639b (127.22.235.195:44217)
I20260808 08:25:05.874109 27209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:05.870446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.194:43833 (local address 127.22.235.193:53441)
0808 08:25:05.870720 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:05.870735 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:05.870857 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:05.871198 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:05.871202 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:05.871223 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:25:05.871343 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:05.871348 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:05.872089 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:05.872093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:05.873007 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:05.873017 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:05.873687 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:05.873690 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:05.873791 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:05.873813 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 08:25:05.873878 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 08:25:05.873951 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":92,"threads_started":1}
I20260808 08:25:05.874100 27211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:05.870554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:53441 (local address 127.22.235.194:43833)
0808 08:25:05.870781 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:05.870784 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:05.870801 (+ 17us) server_negotiation.cc:407] Connection header received
0808 08:25:05.871029 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:05.871033 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:05.871090 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:05.871183 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:25:05.871453 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:05.871975 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:05.873125 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:05.873587 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:05.873695 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:05.873753 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:05.873904 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 08:25:05.873981 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":73,"threads_started":1}
I20260808 08:25:05.874531 26680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3080e977e9e94cbdba3c39b4ee8556ff" candidate_uuid: "4528d69c95104a80aef5b734b2bce43e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f49ac766475b4a0990668efe159a6f7b" is_pre_election: true
I20260808 08:25:05.874694 26680 raft_consensus.cc:2454] T 3080e977e9e94cbdba3c39b4ee8556ff P f49ac766475b4a0990668efe159a6f7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4528d69c95104a80aef5b734b2bce43e in term 0.
I20260808 08:25:05.874759 27200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:05.870996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:35373 (local address 127.22.235.195:44217)
0808 08:25:05.871123 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:05.871127 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:05.871258 (+ 131us) server_negotiation.cc:407] Connection header received
0808 08:25:05.871394 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:05.871397 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:05.871444 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:05.871504 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:25:05.871756 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:05.872299 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:05.873386 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:05.873871 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:05.873980 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:05.874032 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:05.874559 (+ 527us) server_negotiation.cc:299] Negotiation successful
0808 08:25:05.874641 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 08:25:05.874838 27213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:05.870864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.195:44217 (local address 127.22.235.193:35373)
0808 08:25:05.871162 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:05.871173 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:05.871282 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:05.871520 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:05.871523 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:05.871539 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:25:05.871650 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:05.871655 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:05.872419 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:05.872423 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:05.873257 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:05.873268 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:05.873980 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:05.873983 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:05.874083 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:05.874105 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 08:25:05.874522 (+ 417us) client_negotiation.cc:240] Negotiation successful
0808 08:25:05.874618 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"spinlock_wait_cycles":896,"thread_start_us":124,"threads_started":1}
I20260808 08:25:05.875149 26348 leader_election.cc:302] T 3080e977e9e94cbdba3c39b4ee8556ff P 4528d69c95104a80aef5b734b2bce43e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4528d69c95104a80aef5b734b2bce43e, f49ac766475b4a0990668efe159a6f7b; no voters:
I20260808 08:25:05.875403 27188 raft_consensus.cc:2781] T 3080e977e9e94cbdba3c39b4ee8556ff P 4528d69c95104a80aef5b734b2bce43e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 08:25:05.875476 27188 raft_consensus.cc:482] T 3080e977e9e94cbdba3c39b4ee8556ff P 4528d69c95104a80aef5b734b2bce43e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 08:25:05.875531 27188 raft_consensus.cc:3037] T 3080e977e9e94cbdba3c39b4ee8556ff P 4528d69c95104a80aef5b734b2bce43e [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:05.875699 26907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3080e977e9e94cbdba3c39b4ee8556ff" candidate_uuid: "4528d69c95104a80aef5b734b2bce43e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e454ff6295c54d31877c4683dc9a639b" is_pre_election: true
I20260808 08:25:05.875846 26907 raft_consensus.cc:2454] T 3080e977e9e94cbdba3c39b4ee8556ff P e454ff6295c54d31877c4683dc9a639b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4528d69c95104a80aef5b734b2bce43e in term 0.
I20260808 08:25:05.876716 27188 raft_consensus.cc:504] T 3080e977e9e94cbdba3c39b4ee8556ff P 4528d69c95104a80aef5b734b2bce43e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } }
I20260808 08:25:05.877030 27188 leader_election.cc:288] T 3080e977e9e94cbdba3c39b4ee8556ff P 4528d69c95104a80aef5b734b2bce43e [CANDIDATE]: Term 1 election: Requested vote from peers f49ac766475b4a0990668efe159a6f7b (127.22.235.194:43833), e454ff6295c54d31877c4683dc9a639b (127.22.235.195:44217)
I20260808 08:25:05.877400 26680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3080e977e9e94cbdba3c39b4ee8556ff" candidate_uuid: "4528d69c95104a80aef5b734b2bce43e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f49ac766475b4a0990668efe159a6f7b"
I20260808 08:25:05.877517 26680 raft_consensus.cc:3037] T 3080e977e9e94cbdba3c39b4ee8556ff P f49ac766475b4a0990668efe159a6f7b [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:05.878432 26680 raft_consensus.cc:2454] T 3080e977e9e94cbdba3c39b4ee8556ff P f49ac766475b4a0990668efe159a6f7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4528d69c95104a80aef5b734b2bce43e in term 1.
I20260808 08:25:05.878672 26907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3080e977e9e94cbdba3c39b4ee8556ff" candidate_uuid: "4528d69c95104a80aef5b734b2bce43e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e454ff6295c54d31877c4683dc9a639b"
I20260808 08:25:05.878729 26348 leader_election.cc:302] T 3080e977e9e94cbdba3c39b4ee8556ff P 4528d69c95104a80aef5b734b2bce43e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4528d69c95104a80aef5b734b2bce43e, f49ac766475b4a0990668efe159a6f7b; no voters:
I20260808 08:25:05.878778 26907 raft_consensus.cc:3037] T 3080e977e9e94cbdba3c39b4ee8556ff P e454ff6295c54d31877c4683dc9a639b [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:05.878890 27188 raft_consensus.cc:2781] T 3080e977e9e94cbdba3c39b4ee8556ff P 4528d69c95104a80aef5b734b2bce43e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:25:05.879083 27188 raft_consensus.cc:686] T 3080e977e9e94cbdba3c39b4ee8556ff P 4528d69c95104a80aef5b734b2bce43e [term 1 LEADER]: Becoming Leader. State: Replica: 4528d69c95104a80aef5b734b2bce43e, State: Running, Role: LEADER
I20260808 08:25:05.879331 27188 consensus_queue.cc:237] T 3080e977e9e94cbdba3c39b4ee8556ff P 4528d69c95104a80aef5b734b2bce43e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } }
I20260808 08:25:05.879839 26907 raft_consensus.cc:2454] T 3080e977e9e94cbdba3c39b4ee8556ff P e454ff6295c54d31877c4683dc9a639b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4528d69c95104a80aef5b734b2bce43e in term 1.
I20260808 08:25:05.881611 26194 catalog_manager.cc:5495] T 3080e977e9e94cbdba3c39b4ee8556ff P 4528d69c95104a80aef5b734b2bce43e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4528d69c95104a80aef5b734b2bce43e (127.22.235.193). New cstate: current_term: 1 leader_uuid: "4528d69c95104a80aef5b734b2bce43e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4528d69c95104a80aef5b734b2bce43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41383 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 } health_report { overall_health: UNKNOWN } } }
I20260808 08:25:05.897481 27219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:05.894326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48270 (local address 127.22.235.193:41383)
0808 08:25:05.894616 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:05.894622 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:05.894633 (+ 11us) server_negotiation.cc:407] Connection header received
0808 08:25:05.894701 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:05.894703 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:05.894752 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:05.894847 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:25:05.895004 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:05.895517 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:05.896215 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:05.896446 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:05.896517 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:05.896737 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:25:05.897193 (+ 456us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 08:25:05.897278 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:05.897320 (+ 42us) server_negotiation.cc:299] Negotiation successful
0808 08:25:05.897363 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":146,"threads_started":1}
I20260808 08:25:05.906984 27211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:05.903957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:39151 (local address 127.22.235.194:43833)
0808 08:25:05.904074 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:05.904078 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:05.904142 (+ 64us) server_negotiation.cc:407] Connection header received
0808 08:25:05.904245 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:05.904248 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:05.904288 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:05.904347 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:25:05.904655 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:05.905162 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:05.906018 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:05.906226 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:05.906302 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:05.906610 (+ 308us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:25:05.906681 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 08:25:05.906737 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:05.906807 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 08:25:05.906854 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 08:25:05.907002 27224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:05.903733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.194:43833 (local address 127.22.235.193:39151)
0808 08:25:05.904077 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:05.904090 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:05.904174 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:05.904365 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:05.904367 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:05.904374 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 08:25:05.904473 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:05.904478 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:05.905269 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:05.905271 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:05.905895 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:05.905904 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:05.906308 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:05.906312 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:05.906431 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:05.906495 (+ 64us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 08:25:05.906744 (+ 249us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 08:25:05.906748 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 08:25:05.906798 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 08:25:05.906860 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":108,"threads_started":1}
I20260808 08:25:05.912412 26907 raft_consensus.cc:1261] T 3080e977e9e94cbdba3c39b4ee8556ff P e454ff6295c54d31877c4683dc9a639b [term 1 FOLLOWER]: Refusing update from remote peer 4528d69c95104a80aef5b734b2bce43e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:25:05.912500 26680 raft_consensus.cc:1261] T 3080e977e9e94cbdba3c39b4ee8556ff P f49ac766475b4a0990668efe159a6f7b [term 1 FOLLOWER]: Refusing update from remote peer 4528d69c95104a80aef5b734b2bce43e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:25:05.912889 27188 consensus_queue.cc:1035] T 3080e977e9e94cbdba3c39b4ee8556ff P 4528d69c95104a80aef5b734b2bce43e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e454ff6295c54d31877c4683dc9a639b" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:25:05.913087 27216 consensus_queue.cc:1035] T 3080e977e9e94cbdba3c39b4ee8556ff P 4528d69c95104a80aef5b734b2bce43e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f49ac766475b4a0990668efe159a6f7b" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 43833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:25:05.928956 27233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:05.925870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:41383 (local address 127.22.235.194:58083)
0808 08:25:05.926136 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:05.926152 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:05.926248 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:05.926463 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:05.926466 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:05.926474 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 08:25:05.926572 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:05.926577 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:05.927275 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:05.927279 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:05.927869 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:05.927877 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:05.928307 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:05.928311 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:05.928395 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:05.928422 (+ 27us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 08:25:05.928687 (+ 265us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 08:25:05.928694 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 08:25:05.928755 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 08:25:05.928834 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":114,"threads_started":1}
I20260808 08:25:05.928947 27219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:05.925932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:58083 (local address 127.22.235.193:41383)
0808 08:25:05.926154 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:05.926159 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:05.926207 (+ 48us) server_negotiation.cc:407] Connection header received
0808 08:25:05.926336 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:05.926339 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:05.926382 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:05.926446 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:25:05.926661 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:05.927178 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:05.927983 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:05.928212 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:05.928296 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:05.928528 (+ 232us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:25:05.928597 (+ 69us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 08:25:05.928685 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:05.928762 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 08:25:05.928809 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158}
W20260808 08:25:05.929694 27135 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 08:25:07.484267 26192 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:25:07.484423 26192 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33256:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:25:07.486667 26192 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 14fef20eb9444ae995182431d0545751: Sending DeleteTablet for 3 replicas of tablet 3080e977e9e94cbdba3c39b4ee8556ff
I20260808 08:25:07.487161 26415 tablet_service.cc:1514] Processing DeleteTablet for tablet 3080e977e9e94cbdba3c39b4ee8556ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:25:07 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:48258
I20260808 08:25:07.487188 26643 tablet_service.cc:1514] Processing DeleteTablet for tablet 3080e977e9e94cbdba3c39b4ee8556ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:25:07 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:51836
I20260808 08:25:07.487493 27265 tablet_replica.cc:330] T 3080e977e9e94cbdba3c39b4ee8556ff P 4528d69c95104a80aef5b734b2bce43e: stopping tablet replica
I20260808 08:25:07.487507 26872 tablet_service.cc:1514] Processing DeleteTablet for tablet 3080e977e9e94cbdba3c39b4ee8556ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:25:07 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:48792
I20260808 08:25:07.487622 27265 raft_consensus.cc:2229] T 3080e977e9e94cbdba3c39b4ee8556ff P 4528d69c95104a80aef5b734b2bce43e [term 1 LEADER]: Raft consensus shutting down.
I20260808 08:25:07.487648 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 26318
I20260808 08:25:07.510982 27270 tablet_replica.cc:330] T 3080e977e9e94cbdba3c39b4ee8556ff P e454ff6295c54d31877c4683dc9a639b: stopping tablet replica
I20260808 08:25:07.511106 27269 tablet_replica.cc:330] T 3080e977e9e94cbdba3c39b4ee8556ff P f49ac766475b4a0990668efe159a6f7b: stopping tablet replica
I20260808 08:25:07.511161 27270 raft_consensus.cc:2229] T 3080e977e9e94cbdba3c39b4ee8556ff P e454ff6295c54d31877c4683dc9a639b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 08:25:07.511273 27269 raft_consensus.cc:2229] T 3080e977e9e94cbdba3c39b4ee8556ff P f49ac766475b4a0990668efe159a6f7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 08:25:07.511310 27270 raft_consensus.cc:2258] T 3080e977e9e94cbdba3c39b4ee8556ff P e454ff6295c54d31877c4683dc9a639b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:25:07.511405 27269 raft_consensus.cc:2258] T 3080e977e9e94cbdba3c39b4ee8556ff P f49ac766475b4a0990668efe159a6f7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:25:07.512089 27270 ts_tablet_manager.cc:1905] T 3080e977e9e94cbdba3c39b4ee8556ff P e454ff6295c54d31877c4683dc9a639b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:25:07.513100 27269 ts_tablet_manager.cc:1905] T 3080e977e9e94cbdba3c39b4ee8556ff P f49ac766475b4a0990668efe159a6f7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:25:07.514981 27270 ts_tablet_manager.cc:1918] T 3080e977e9e94cbdba3c39b4ee8556ff P e454ff6295c54d31877c4683dc9a639b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 08:25:07.515081 27270 log.cc:1196] T 3080e977e9e94cbdba3c39b4ee8556ff P e454ff6295c54d31877c4683dc9a639b: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-2/wal/wals/3080e977e9e94cbdba3c39b4ee8556ff
I20260808 08:25:07.515301 27269 ts_tablet_manager.cc:1918] T 3080e977e9e94cbdba3c39b4ee8556ff P f49ac766475b4a0990668efe159a6f7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 08:25:07.515326 27270 ts_tablet_manager.cc:1939] T 3080e977e9e94cbdba3c39b4ee8556ff P e454ff6295c54d31877c4683dc9a639b: Deleting consensus metadata
I20260808 08:25:07.515373 27269 log.cc:1196] T 3080e977e9e94cbdba3c39b4ee8556ff P f49ac766475b4a0990668efe159a6f7b: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786177493204855-23471-0/minicluster-data/ts-1/wal/wals/3080e977e9e94cbdba3c39b4ee8556ff
I20260808 08:25:07.515595 27269 ts_tablet_manager.cc:1939] T 3080e977e9e94cbdba3c39b4ee8556ff P f49ac766475b4a0990668efe159a6f7b: Deleting consensus metadata
I20260808 08:25:07.516203 26172 catalog_manager.cc:4841] TS e454ff6295c54d31877c4683dc9a639b (127.22.235.195:44217): tablet 3080e977e9e94cbdba3c39b4ee8556ff (table test-table [id=24098858f8d94d5cad494a9aaf7b7ba3]) successfully deleted
I20260808 08:25:07.516386 26172 catalog_manager.cc:4841] TS f49ac766475b4a0990668efe159a6f7b (127.22.235.194:43833): tablet 3080e977e9e94cbdba3c39b4ee8556ff (table test-table [id=24098858f8d94d5cad494a9aaf7b7ba3]) successfully deleted
W20260808 08:25:07.519675 26170 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.235.193:41383 (error 108)
I20260808 08:25:07.519896 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 26536
I20260808 08:25:07.528996 27275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:07.528402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:41383 (local address 127.0.0.1:48280)
0808 08:25:07.528799 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:07.528887 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.193:41383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":98,"threads_started":1}
W20260808 08:25:07.529119 26170 catalog_manager.cc:4584] TS 4528d69c95104a80aef5b734b2bce43e (127.22.235.193:41383): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3080e977e9e94cbdba3c39b4ee8556ff: Network error: Client connection negotiation failed: client connection to 127.22.235.193:41383: connect: Connection refused (error 111)
I20260808 08:25:07.538394 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 26764
I20260808 08:25:07.560708 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 26143
2026-08-08T08:25:07Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6757 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 08:25:07 dist-test-slave-f7th krb5kdc[27279](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 08:25:07 dist-test-slave-f7th krb5kdc[27279](info): set up 2 sockets
Aug 08 08:25:07 dist-test-slave-f7th krb5kdc[27279](info): commencing operation
krb5kdc: starting...
W20260808 08:25:08.741019 23471 mini_kdc.cc:121] Time spent starting KDC: real 1.145s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Aug 08 08:25:08 dist-test-slave-f7th krb5kdc[27279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177508, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T08:25:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:25:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.235.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
I20260808 08:25:08.863107 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:35627
--webserver_interface=127.22.235.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:36617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:35627
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:25:08.975440 27299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:08.975692 27299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:08.975730 27299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:08.979635 27299 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:25:08.979691 27299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:08.979709 27299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:08.979724 27299 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:25:08.979740 27299 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:25:08.984907 27299 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:36617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:35627
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:35627
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.235.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27299
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:08.985952 27299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:08.986985 27299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 08:25:08.995529 27299 server_base.cc:1034] running on GCE node
W20260808 08:25:08.995440 27310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:08.995550 27311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:08.995440 27318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:08.996245 27299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:08.997385 27299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:08.998555 27299 hybrid_clock.cc:648] HybridClock initialized: now 1786177508998542 us; error 24 us; skew 500 ppm
Aug 08 08:25:08 dist-test-slave-f7th krb5kdc[27279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177508, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:09.001348 27299 init.cc:377] Logged in from keytab as kudu/127.22.235.254@KRBTEST.COM (short username kudu)
I20260808 08:25:09.002192 27299 webserver.cc:460] Webserver started at http://127.22.235.254:34133/ using document root <none> and password file <none>
I20260808 08:25:09.002725 27299 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:09.002791 27299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:09.003039 27299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:09.005105 27299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "2d00befe6ad94655bce6d645c2eeca24"
format_stamp: "Formatted at 2026-08-08 08:25:08 on dist-test-slave-f7th"
server_key: "35ff35ff497c07ccf69a25e7d7131791"
server_key_iv: "58859abdd23231ef179486d91cc09f1e"
server_key_version: "encryptionkey@0"
I20260808 08:25:09.005645 27299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "2d00befe6ad94655bce6d645c2eeca24"
format_stamp: "Formatted at 2026-08-08 08:25:08 on dist-test-slave-f7th"
server_key: "35ff35ff497c07ccf69a25e7d7131791"
server_key_iv: "58859abdd23231ef179486d91cc09f1e"
server_key_version: "encryptionkey@0"
I20260808 08:25:09.009280 27299 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 08:25:09.011570 27334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:09.012848 27299 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:25:09.013001 27299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "2d00befe6ad94655bce6d645c2eeca24"
format_stamp: "Formatted at 2026-08-08 08:25:08 on dist-test-slave-f7th"
server_key: "35ff35ff497c07ccf69a25e7d7131791"
server_key_iv: "58859abdd23231ef179486d91cc09f1e"
server_key_version: "encryptionkey@0"
I20260808 08:25:09.013093 27299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:09.025718 27299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:09.028380 27299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:09.028556 27299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:09.036381 27299 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.254:35627
I20260808 08:25:09.036388 27425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.254:35627 every 8 connection(s)
I20260808 08:25:09.037353 27299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:25:09.039863 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 27299
I20260808 08:25:09.039956 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:25:09.040158 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:09.040206 23471 external_mini_cluster.cc:1428] Setting key 1fd51fd563562de6dcb00fcdfd393dbb
I20260808 08:25:09.047259 27426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d00befe6ad94655bce6d645c2eeca24: Bootstrap starting.
Aug 08 08:25:09 dist-test-slave-f7th krb5kdc[27279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177508, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:25:09.049818 27426 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d00befe6ad94655bce6d645c2eeca24: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:09.050630 27426 log.cc:824] T 00000000000000000000000000000000 P 2d00befe6ad94655bce6d645c2eeca24: Log is configured to *not* fsync() on all Append() calls
I20260808 08:25:09.052870 27426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d00befe6ad94655bce6d645c2eeca24: No bootstrap required, opened a new log
I20260808 08:25:09.055277 27429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:09.041647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48980 (local address 127.22.235.254:35627)
0808 08:25:09.042049 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:09.042068 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:09.042111 (+ 43us) server_negotiation.cc:407] Connection header received
0808 08:25:09.042842 (+ 731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:09.042866 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:09.043286 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:09.043658 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:09.044314 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:09.045808 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:09.046666 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:09.047341 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:09.047518 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:09.050227 (+ 2709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:09.050233 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:09.050250 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:09.050280 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:09.052217 (+ 1937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:09.052701 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:09.052708 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:09.052714 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:09.052849 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:09.053138 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:09.053141 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:09.053143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:09.053641 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:09.053826 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:09.054041 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 08:25:09.054332 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":109,"threads_started":1}
I20260808 08:25:09.057176 27426 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d00befe6ad94655bce6d645c2eeca24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d00befe6ad94655bce6d645c2eeca24" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 35627 } }
I20260808 08:25:09.057411 27426 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d00befe6ad94655bce6d645c2eeca24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:09.057515 27426 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d00befe6ad94655bce6d645c2eeca24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d00befe6ad94655bce6d645c2eeca24, State: Initialized, Role: FOLLOWER
I20260808 08:25:09.057947 27426 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d00befe6ad94655bce6d645c2eeca24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d00befe6ad94655bce6d645c2eeca24" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 35627 } }
I20260808 08:25:09.058079 27426 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d00befe6ad94655bce6d645c2eeca24 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:25:09.058136 27426 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d00befe6ad94655bce6d645c2eeca24 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:25:09.058233 27426 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d00befe6ad94655bce6d645c2eeca24 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:09.059074 27426 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d00befe6ad94655bce6d645c2eeca24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d00befe6ad94655bce6d645c2eeca24" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 35627 } }
I20260808 08:25:09.059386 27426 leader_election.cc:302] T 00000000000000000000000000000000 P 2d00befe6ad94655bce6d645c2eeca24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d00befe6ad94655bce6d645c2eeca24; no voters:
I20260808 08:25:09.059652 27426 leader_election.cc:288] T 00000000000000000000000000000000 P 2d00befe6ad94655bce6d645c2eeca24 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:25:09.059778 27432 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d00befe6ad94655bce6d645c2eeca24 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:25:09.060019 27432 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d00befe6ad94655bce6d645c2eeca24 [term 1 LEADER]: Becoming Leader. State: Replica: 2d00befe6ad94655bce6d645c2eeca24, State: Running, Role: LEADER
I20260808 08:25:09.060323 27432 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d00befe6ad94655bce6d645c2eeca24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d00befe6ad94655bce6d645c2eeca24" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 35627 } }
I20260808 08:25:09.060822 27426 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d00befe6ad94655bce6d645c2eeca24 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:25:09.061951 27437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d00befe6ad94655bce6d645c2eeca24 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d00befe6ad94655bce6d645c2eeca24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d00befe6ad94655bce6d645c2eeca24" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 35627 } } }
I20260808 08:25:09.062079 27437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d00befe6ad94655bce6d645c2eeca24 [sys.catalog]: This master's current role is: LEADER
I20260808 08:25:09.062481 27450 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:25:09.062462 27438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d00befe6ad94655bce6d645c2eeca24 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d00befe6ad94655bce6d645c2eeca24. Latest consensus state: current_term: 1 leader_uuid: "2d00befe6ad94655bce6d645c2eeca24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d00befe6ad94655bce6d645c2eeca24" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 35627 } } }
I20260808 08:25:09.062678 27438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d00befe6ad94655bce6d645c2eeca24 [sys.catalog]: This master's current role is: LEADER
I20260808 08:25:09.065863 27450 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:25:09.071197 27450 catalog_manager.cc:1347] Generated new cluster ID: 06793f08e4334b7e8f829ff2afc843d1
I20260808 08:25:09.071260 27450 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:25:09.094317 27450 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:25:09.095707 27450 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:25:09.104029 27450 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d00befe6ad94655bce6d645c2eeca24: Generated new TSK 0
I20260808 08:25:09.104861 27450 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.235.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab.
Entry for principal kudu/127.22.235.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab.
I20260808 08:25:09.143653 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.193:0
--local_ip_for_outbound_sockets=127.22.235.193
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:35627
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab
--principal=kudu/127.22.235.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:36617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:25:09.257498 27472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:09.257738 27472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:09.257786 27472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:09.261808 27472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:09.261865 27472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:09.261934 27472 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.193
I20260808 08:25:09.267170 27472 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:36617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab
--principal=kudu/127.22.235.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:35627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.193
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:09.268234 27472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:09.269352 27472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:09.278718 27485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:09.278813 27472 server_base.cc:1034] running on GCE node
W20260808 08:25:09.278728 27483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:09.278718 27488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:09.279317 27472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:09.280053 27472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:09.281252 27472 hybrid_clock.cc:648] HybridClock initialized: now 1786177509281227 us; error 42 us; skew 500 ppm
Aug 08 08:25:09 dist-test-slave-f7th krb5kdc[27279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177509, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:09.283913 27472 init.cc:377] Logged in from keytab as kudu/127.22.235.193@KRBTEST.COM (short username kudu)
I20260808 08:25:09.284897 27472 webserver.cc:460] Webserver started at http://127.22.235.193:42687/ using document root <none> and password file <none>
I20260808 08:25:09.285437 27472 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:09.285502 27472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:09.285692 27472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:09.287676 27472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-0/data/instance:
uuid: "434fb6e5637d40d5a181b192d90ac485"
format_stamp: "Formatted at 2026-08-08 08:25:09 on dist-test-slave-f7th"
server_key: "9970e8d3bed9841f58dcf2f69b0b2937"
server_key_iv: "7361a2e213c1a0dd79854bfce6a383f0"
server_key_version: "encryptionkey@0"
I20260808 08:25:09.288213 27472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance:
uuid: "434fb6e5637d40d5a181b192d90ac485"
format_stamp: "Formatted at 2026-08-08 08:25:09 on dist-test-slave-f7th"
server_key: "9970e8d3bed9841f58dcf2f69b0b2937"
server_key_iv: "7361a2e213c1a0dd79854bfce6a383f0"
server_key_version: "encryptionkey@0"
I20260808 08:25:09.291791 27472 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 08:25:09.294163 27497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:09.295285 27472 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 08:25:09.295392 27472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-0/wal
uuid: "434fb6e5637d40d5a181b192d90ac485"
format_stamp: "Formatted at 2026-08-08 08:25:09 on dist-test-slave-f7th"
server_key: "9970e8d3bed9841f58dcf2f69b0b2937"
server_key_iv: "7361a2e213c1a0dd79854bfce6a383f0"
server_key_version: "encryptionkey@0"
I20260808 08:25:09.295481 27472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:09.310240 27472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:09.313072 27472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:09.313249 27472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:09.313818 27472 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:25:09.314626 27472 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:25:09.314671 27472 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:09.314713 27472 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:25:09.314741 27472 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:09.323045 27472 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.193:36993
I20260808 08:25:09.323113 27698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.193:36993 every 8 connection(s)
I20260808 08:25:09.324182 27472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
I20260808 08:25:09.330078 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 27472
I20260808 08:25:09.330192 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance
I20260808 08:25:09.330446 23471 external_mini_cluster.cc:1428] Setting key b35ac2f994f3ae3572f6d8dcb121031d
Aug 08 08:25:09 dist-test-slave-f7th krb5kdc[27279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177509, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:25:09.337476 27429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:09.325987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:36937 (local address 127.22.235.254:35627)
0808 08:25:09.326145 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:09.326149 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:09.327156 (+ 1007us) server_negotiation.cc:407] Connection header received
0808 08:25:09.328028 (+ 872us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:09.328032 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:09.328091 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:09.328165 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:09.329393 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:09.329990 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:09.331342 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:09.331584 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:09.331677 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:09.334099 (+ 2422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:09.334103 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:09.334106 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:09.334135 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:09.335460 (+ 1325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:09.335927 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:09.335931 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:09.335933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:09.336025 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:09.336340 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:09.336344 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:09.336346 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:09.336533 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:09.336647 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:09.337171 (+ 524us) server_negotiation.cc:299] Negotiation successful
0808 08:25:09.337305 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":53}
I20260808 08:25:09.338331 27705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:09.326351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:35627 (local address 127.22.235.193:36937)
0808 08:25:09.327011 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:09.327053 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:09.327827 (+ 774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:09.328346 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:09.328353 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:09.328745 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:09.329233 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:09.329247 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:09.330135 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:09.330138 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:09.331208 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:09.331221 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:09.331741 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:09.331746 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:09.331944 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:09.332513 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:09.332530 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:09.333934 (+ 1404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:09.335562 (+ 1628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:09.335568 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:09.335580 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:09.335821 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:09.336113 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:09.336116 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:09.336117 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:09.336249 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:09.336664 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:09.336670 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:09.336921 (+ 251us) client_negotiation.cc:769] Sending connection context
0808 08:25:09.337112 (+ 191us) client_negotiation.cc:240] Negotiation successful
0808 08:25:09.337460 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":119,"threads_started":1}
I20260808 08:25:09.339480 27701 heartbeater.cc:347] Connected to a master server at 127.22.235.254:35627
I20260808 08:25:09.339712 27701 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:09.340195 27701 heartbeater.cc:511] Master 127.22.235.254:35627 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.22.235.194@KRBTEST.COM; defaulting to no policy
I20260808 08:25:09.341727 27364 ts_manager.cc:194] Registered new tserver with Master: 434fb6e5637d40d5a181b192d90ac485 (127.22.235.193:36993)
I20260808 08:25:09.342854 27364 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.193@KRBTEST.COM'} at 127.22.235.193:36937
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab.
Entry for principal kudu/127.22.235.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab.
I20260808 08:25:09.357205 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.194:0
--local_ip_for_outbound_sockets=127.22.235.194
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:35627
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab
--principal=kudu/127.22.235.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:36617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:25:09.468168 27710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:09.468405 27710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:09.468446 27710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:09.472855 27710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:09.472906 27710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:09.472971 27710 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.194
I20260808 08:25:09.478271 27710 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:36617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab
--principal=kudu/127.22.235.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:35627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27710
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.194
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:09.479228 27710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:09.480196 27710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:09.489611 27730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:09.489683 27720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:09.489864 27710 server_base.cc:1034] running on GCE node
W20260808 08:25:09.489935 27721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:09.490301 27710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:09.490958 27710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:09.492129 27710 hybrid_clock.cc:648] HybridClock initialized: now 1786177509492108 us; error 28 us; skew 500 ppm
Aug 08 08:25:09 dist-test-slave-f7th krb5kdc[27279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177509, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:09.494844 27710 init.cc:377] Logged in from keytab as kudu/127.22.235.194@KRBTEST.COM (short username kudu)
I20260808 08:25:09.495711 27710 webserver.cc:460] Webserver started at http://127.22.235.194:42419/ using document root <none> and password file <none>
I20260808 08:25:09.496271 27710 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:09.496342 27710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:09.496551 27710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:09.498582 27710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-1/data/instance:
uuid: "0eca654088434614aec8fdf7631082b2"
format_stamp: "Formatted at 2026-08-08 08:25:09 on dist-test-slave-f7th"
server_key: "e0ccebe375c08e5e044040f1abff8027"
server_key_iv: "1ff678d89750579648b4f5c30cf2edd9"
server_key_version: "encryptionkey@0"
I20260808 08:25:09.499148 27710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance:
uuid: "0eca654088434614aec8fdf7631082b2"
format_stamp: "Formatted at 2026-08-08 08:25:09 on dist-test-slave-f7th"
server_key: "e0ccebe375c08e5e044040f1abff8027"
server_key_iv: "1ff678d89750579648b4f5c30cf2edd9"
server_key_version: "encryptionkey@0"
I20260808 08:25:09.502964 27710 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 08:25:09.505440 27738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:09.506600 27710 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:25:09.506709 27710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-1/wal
uuid: "0eca654088434614aec8fdf7631082b2"
format_stamp: "Formatted at 2026-08-08 08:25:09 on dist-test-slave-f7th"
server_key: "e0ccebe375c08e5e044040f1abff8027"
server_key_iv: "1ff678d89750579648b4f5c30cf2edd9"
server_key_version: "encryptionkey@0"
I20260808 08:25:09.506816 27710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:09.524111 27710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:09.526731 27710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:09.526929 27710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:09.527491 27710 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:25:09.528309 27710 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:25:09.528357 27710 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:09.528412 27710 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:25:09.528440 27710 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:09.537221 27710 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.194:45293
I20260808 08:25:09.537251 27929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.194:45293 every 8 connection(s)
I20260808 08:25:09.538081 27710 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
I20260808 08:25:09.542826 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 27710
I20260808 08:25:09.542945 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance
I20260808 08:25:09.543202 23471 external_mini_cluster.cc:1428] Setting key cae6c1c95feaa4742e6a6adb81d5aa0d
Aug 08 08:25:09 dist-test-slave-f7th krb5kdc[27279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177509, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.194@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:25:09.551553 27429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:09.539799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:38059 (local address 127.22.235.254:35627)
0808 08:25:09.539945 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:09.539949 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:09.540980 (+ 1031us) server_negotiation.cc:407] Connection header received
0808 08:25:09.541846 (+ 866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:09.541850 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:09.541908 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:09.541996 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:09.543225 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:09.543794 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:09.545228 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:09.545524 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:09.545630 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:09.548161 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:09.548164 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:09.548166 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:09.548191 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:09.549488 (+ 1297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:09.550064 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:09.550067 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:09.550068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:09.550131 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:09.550481 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:09.550483 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:09.550484 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:09.550648 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:09.550715 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:09.551320 (+ 605us) server_negotiation.cc:299] Negotiation successful
0808 08:25:09.551411 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 08:25:09.552743 27932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:09.540174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:35627 (local address 127.22.235.194:38059)
0808 08:25:09.540826 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:09.540867 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:09.541640 (+ 773us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:09.542194 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:09.542202 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:09.542552 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:09.543077 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:09.543093 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:09.543959 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:09.543962 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:09.545106 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:09.545116 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:09.545654 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:09.545657 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:09.545788 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:09.546323 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:09.546339 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:09.547996 (+ 1657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:09.549604 (+ 1608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:09.549611 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:09.549626 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:09.549977 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:09.550211 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:09.550214 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:09.550216 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:09.550401 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:09.550742 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:09.550748 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:09.551083 (+ 335us) client_negotiation.cc:769] Sending connection context
0808 08:25:09.551306 (+ 223us) client_negotiation.cc:240] Negotiation successful
0808 08:25:09.551655 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":99,"threads_started":1}
WARNING: no policy specified for kudu/127.22.235.195@KRBTEST.COM; defaulting to no policy
I20260808 08:25:09.553890 27930 heartbeater.cc:347] Connected to a master server at 127.22.235.254:35627
I20260808 08:25:09.554131 27930 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:09.554597 27930 heartbeater.cc:511] Master 127.22.235.254:35627 requested a full tablet report, sending...
I20260808 08:25:09.555634 27364 ts_manager.cc:194] Registered new tserver with Master: 0eca654088434614aec8fdf7631082b2 (127.22.235.194:45293)
I20260808 08:25:09.556218 27364 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.194@KRBTEST.COM'} at 127.22.235.194:38059
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab.
Entry for principal kudu/127.22.235.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab.
I20260808 08:25:09.571403 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.195:0
--local_ip_for_outbound_sockets=127.22.235.195
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:35627
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab
--principal=kudu/127.22.235.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:36617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:25:09.685017 27937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:09.685269 27937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:09.685329 27937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:09.689272 27937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:09.689324 27937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:09.689401 27937 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.195
I20260808 08:25:09.694844 27937 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:36617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab
--principal=kudu/127.22.235.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:35627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27937
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.195
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:09.695856 27937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:09.696923 27937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 08:25:09.706568 27937 server_base.cc:1034] running on GCE node
W20260808 08:25:09.706476 27947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:09.706496 27950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:09.706476 27948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:09.707217 27937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:09.707834 27937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:09.708990 27937 hybrid_clock.cc:648] HybridClock initialized: now 1786177509708969 us; error 33 us; skew 500 ppm
Aug 08 08:25:09 dist-test-slave-f7th krb5kdc[27279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177509, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:09.711700 27937 init.cc:377] Logged in from keytab as kudu/127.22.235.195@KRBTEST.COM (short username kudu)
I20260808 08:25:09.712502 27937 webserver.cc:460] Webserver started at http://127.22.235.195:40397/ using document root <none> and password file <none>
I20260808 08:25:09.713045 27937 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:09.713105 27937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:09.713272 27937 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:09.715265 27937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-2/data/instance:
uuid: "27bfb872caef49d8a46c91873d59d159"
format_stamp: "Formatted at 2026-08-08 08:25:09 on dist-test-slave-f7th"
server_key: "783273e9493abc7487902a18a4eaa977"
server_key_iv: "c4fe123ec50772d5987839270b0e4a1b"
server_key_version: "encryptionkey@0"
I20260808 08:25:09.715818 27937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance:
uuid: "27bfb872caef49d8a46c91873d59d159"
format_stamp: "Formatted at 2026-08-08 08:25:09 on dist-test-slave-f7th"
server_key: "783273e9493abc7487902a18a4eaa977"
server_key_iv: "c4fe123ec50772d5987839270b0e4a1b"
server_key_version: "encryptionkey@0"
I20260808 08:25:09.719594 27937 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 08:25:09.722015 27960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:09.723151 27937 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 08:25:09.723261 27937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-2/wal
uuid: "27bfb872caef49d8a46c91873d59d159"
format_stamp: "Formatted at 2026-08-08 08:25:09 on dist-test-slave-f7th"
server_key: "783273e9493abc7487902a18a4eaa977"
server_key_iv: "c4fe123ec50772d5987839270b0e4a1b"
server_key_version: "encryptionkey@0"
I20260808 08:25:09.723343 27937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:09.747757 27937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:09.750505 27937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:09.750659 27937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:09.751204 27937 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:25:09.752027 27937 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:25:09.752074 27937 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:09.752112 27937 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:25:09.752132 27937 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:09.760736 27937 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.195:43247
I20260808 08:25:09.760766 28154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.195:43247 every 8 connection(s)
I20260808 08:25:09.761672 27937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
I20260808 08:25:09.767544 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 27937
I20260808 08:25:09.767673 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance
I20260808 08:25:09.767992 23471 external_mini_cluster.cc:1428] Setting key 521859c36310965eadba00328ec0835d
Aug 08 08:25:09 dist-test-slave-f7th krb5kdc[27279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177509, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.195@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:25:09.774525 27429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:09.763576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:38513 (local address 127.22.235.254:35627)
0808 08:25:09.763716 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:09.763720 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:09.764757 (+ 1037us) server_negotiation.cc:407] Connection header received
0808 08:25:09.765603 (+ 846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:09.765606 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:09.765654 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:09.765727 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:09.766831 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:09.767354 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:09.768892 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:09.769149 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:09.769244 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:09.771616 (+ 2372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:09.771620 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:09.771622 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:09.771648 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:09.772846 (+ 1198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:09.773295 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:09.773297 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:09.773298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:09.773367 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:09.773617 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:09.773619 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:09.773620 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:09.773787 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:09.773861 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:09.774291 (+ 430us) server_negotiation.cc:299] Negotiation successful
0808 08:25:09.774388 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 08:25:09.775365 28158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:09.763928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:35627 (local address 127.22.235.195:38513)
0808 08:25:09.764581 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:09.764649 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:09.765428 (+ 779us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:09.765875 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:09.765882 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:09.766239 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:09.766715 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:09.766729 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:09.767495 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:09.767498 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:09.768769 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:09.768781 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:09.769298 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:09.769302 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:09.769469 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:09.770025 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:09.770042 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:09.771450 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:09.772952 (+ 1502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:09.772957 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:09.772969 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:09.773223 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:09.773428 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:09.773429 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:09.773431 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:09.773553 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:09.773856 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:09.773861 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:09.774100 (+ 239us) client_negotiation.cc:769] Sending connection context
0808 08:25:09.774259 (+ 159us) client_negotiation.cc:240] Negotiation successful
0808 08:25:09.774513 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":116,"threads_started":1}
I20260808 08:25:09.776486 28155 heartbeater.cc:347] Connected to a master server at 127.22.235.254:35627
I20260808 08:25:09.776736 28155 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:09.777221 28155 heartbeater.cc:511] Master 127.22.235.254:35627 requested a full tablet report, sending...
I20260808 08:25:09.778265 27364 ts_manager.cc:194] Registered new tserver with Master: 27bfb872caef49d8a46c91873d59d159 (127.22.235.195:43247)
I20260808 08:25:09.778811 27364 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.195@KRBTEST.COM'} at 127.22.235.195:38513
I20260808 08:25:09.781836 23471 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 08:25:09.795768 27429 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:09.794446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48996 (local address 127.22.235.254:35627)
0808 08:25:09.794609 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:09.794613 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:09.794697 (+ 84us) server_negotiation.cc:407] Connection header received
0808 08:25:09.794817 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:09.794819 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:09.794862 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:09.794927 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:09.795630 (+ 703us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48996: BlockingRecv error: recv got EOF from 127.0.0.1:48996 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 08:25:09.796909 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 27472
I20260808 08:25:09.815776 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 27710
I20260808 08:25:09.829002 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 27937
I20260808 08:25:09.853583 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 27299
2026-08-08T08:25:09Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2285 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786177493204855-23471-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 08:25:09 dist-test-slave-f7th krb5kdc[28179](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 08:25:09 dist-test-slave-f7th krb5kdc[28179](info): set up 2 sockets
Aug 08 08:25:09 dist-test-slave-f7th krb5kdc[28179](info): commencing operation
krb5kdc: starting...
W20260808 08:25:11.016031 23471 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Aug 08 08:25:11 dist-test-slave-f7th krb5kdc[28179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177511, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T08:25:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:25:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.235.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
I20260808 08:25:11.134927 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:41023
--webserver_interface=127.22.235.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:43331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:41023
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:25:11.247830 28200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:11.248190 28200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:11.248252 28200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:11.253504 28200 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:25:11.253574 28200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:11.253594 28200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:11.253609 28200 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:25:11.253623 28200 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:25:11.258968 28200 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:43331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:41023
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:41023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.235.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28200
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:11.260025 28200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:11.261094 28200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:11.269286 28207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:11.269351 28200 server_base.cc:1034] running on GCE node
W20260808 08:25:11.269307 28213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:11.269310 28211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:11.270138 28200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:11.271214 28200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:11.272353 28200 hybrid_clock.cc:648] HybridClock initialized: now 1786177511272338 us; error 28 us; skew 500 ppm
Aug 08 08:25:11 dist-test-slave-f7th krb5kdc[28179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177511, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:11.279268 28200 init.cc:377] Logged in from keytab as kudu/127.22.235.254@KRBTEST.COM (short username kudu)
I20260808 08:25:11.280126 28200 webserver.cc:460] Webserver started at http://127.22.235.254:37079/ using document root <none> and password file <none>
I20260808 08:25:11.280691 28200 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:11.280757 28200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:11.280943 28200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:11.282919 28200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "293d7890d3b74d37801cec27a4f65bdb"
format_stamp: "Formatted at 2026-08-08 08:25:11 on dist-test-slave-f7th"
server_key: "ab38a48291dad6317811849d096bd021"
server_key_iv: "edd5ec0cf525125875da553582a61620"
server_key_version: "encryptionkey@0"
I20260808 08:25:11.283428 28200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "293d7890d3b74d37801cec27a4f65bdb"
format_stamp: "Formatted at 2026-08-08 08:25:11 on dist-test-slave-f7th"
server_key: "ab38a48291dad6317811849d096bd021"
server_key_iv: "edd5ec0cf525125875da553582a61620"
server_key_version: "encryptionkey@0"
I20260808 08:25:11.287052 28200 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 08:25:11.289389 28222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:11.290601 28200 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:25:11.290719 28200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "293d7890d3b74d37801cec27a4f65bdb"
format_stamp: "Formatted at 2026-08-08 08:25:11 on dist-test-slave-f7th"
server_key: "ab38a48291dad6317811849d096bd021"
server_key_iv: "edd5ec0cf525125875da553582a61620"
server_key_version: "encryptionkey@0"
I20260808 08:25:11.290813 28200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:11.313957 28200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:11.316781 28200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:11.316967 28200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:11.323776 28200 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.254:41023
I20260808 08:25:11.324030 28315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.254:41023 every 8 connection(s)
I20260808 08:25:11.324765 28200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:25:11.327585 28316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:11.330838 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 28200
I20260808 08:25:11.330950 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:25:11.331197 23471 external_mini_cluster.cc:1428] Setting key 81128ea8bbf0fc1b523baeb72341fa0b
I20260808 08:25:11.333428 28316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 293d7890d3b74d37801cec27a4f65bdb: Bootstrap starting.
I20260808 08:25:11.335880 28316 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 293d7890d3b74d37801cec27a4f65bdb: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:11.336764 28316 log.cc:824] T 00000000000000000000000000000000 P 293d7890d3b74d37801cec27a4f65bdb: Log is configured to *not* fsync() on all Append() calls
I20260808 08:25:11.338698 28316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 293d7890d3b74d37801cec27a4f65bdb: No bootstrap required, opened a new log
Aug 08 08:25:11 dist-test-slave-f7th krb5kdc[28179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177511, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:25:11.342459 28316 raft_consensus.cc:348] T 00000000000000000000000000000000 P 293d7890d3b74d37801cec27a4f65bdb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "293d7890d3b74d37801cec27a4f65bdb" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 41023 } }
I20260808 08:25:11.342732 28316 raft_consensus.cc:374] T 00000000000000000000000000000000 P 293d7890d3b74d37801cec27a4f65bdb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:11.342810 28316 raft_consensus.cc:729] T 00000000000000000000000000000000 P 293d7890d3b74d37801cec27a4f65bdb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 293d7890d3b74d37801cec27a4f65bdb, State: Initialized, Role: FOLLOWER
I20260808 08:25:11.343286 28316 consensus_queue.cc:260] T 00000000000000000000000000000000 P 293d7890d3b74d37801cec27a4f65bdb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "293d7890d3b74d37801cec27a4f65bdb" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 41023 } }
I20260808 08:25:11.343425 28316 raft_consensus.cc:388] T 00000000000000000000000000000000 P 293d7890d3b74d37801cec27a4f65bdb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:25:11.343487 28316 raft_consensus.cc:482] T 00000000000000000000000000000000 P 293d7890d3b74d37801cec27a4f65bdb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:25:11.343585 28316 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 293d7890d3b74d37801cec27a4f65bdb [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:11.344485 28316 raft_consensus.cc:504] T 00000000000000000000000000000000 P 293d7890d3b74d37801cec27a4f65bdb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "293d7890d3b74d37801cec27a4f65bdb" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 41023 } }
I20260808 08:25:11.344837 28316 leader_election.cc:302] T 00000000000000000000000000000000 P 293d7890d3b74d37801cec27a4f65bdb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 293d7890d3b74d37801cec27a4f65bdb; no voters:
I20260808 08:25:11.345116 28316 leader_election.cc:288] T 00000000000000000000000000000000 P 293d7890d3b74d37801cec27a4f65bdb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:25:11.345222 28328 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 293d7890d3b74d37801cec27a4f65bdb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:25:11.345247 28322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:11.332453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38140 (local address 127.22.235.254:41023)
0808 08:25:11.332885 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:11.332902 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:11.332938 (+ 36us) server_negotiation.cc:407] Connection header received
0808 08:25:11.333579 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:11.333598 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:11.333933 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:11.334208 (+ 275us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:11.335103 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:11.336103 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:11.337270 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:11.337822 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:11.337989 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:11.340672 (+ 2683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:11.340678 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:11.340692 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:11.340720 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:11.342264 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:11.342739 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:11.342746 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:11.342751 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:11.342886 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:11.343163 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:11.343165 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:11.343167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:11.343607 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:11.343807 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:11.344098 (+ 291us) server_negotiation.cc:299] Negotiation successful
0808 08:25:11.344372 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":136,"threads_started":1}
I20260808 08:25:11.345444 28328 raft_consensus.cc:686] T 00000000000000000000000000000000 P 293d7890d3b74d37801cec27a4f65bdb [term 1 LEADER]: Becoming Leader. State: Replica: 293d7890d3b74d37801cec27a4f65bdb, State: Running, Role: LEADER
I20260808 08:25:11.345758 28328 consensus_queue.cc:237] T 00000000000000000000000000000000 P 293d7890d3b74d37801cec27a4f65bdb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "293d7890d3b74d37801cec27a4f65bdb" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 41023 } }
I20260808 08:25:11.346335 28316 sys_catalog.cc:564] T 00000000000000000000000000000000 P 293d7890d3b74d37801cec27a4f65bdb [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:25:11.348186 28331 sys_catalog.cc:455] T 00000000000000000000000000000000 P 293d7890d3b74d37801cec27a4f65bdb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 293d7890d3b74d37801cec27a4f65bdb. Latest consensus state: current_term: 1 leader_uuid: "293d7890d3b74d37801cec27a4f65bdb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "293d7890d3b74d37801cec27a4f65bdb" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 41023 } } }
I20260808 08:25:11.348305 28331 sys_catalog.cc:458] T 00000000000000000000000000000000 P 293d7890d3b74d37801cec27a4f65bdb [sys.catalog]: This master's current role is: LEADER
I20260808 08:25:11.348721 28352 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:25:11.348937 23471 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 08:25:11.349030 28329 sys_catalog.cc:455] T 00000000000000000000000000000000 P 293d7890d3b74d37801cec27a4f65bdb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "293d7890d3b74d37801cec27a4f65bdb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "293d7890d3b74d37801cec27a4f65bdb" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 41023 } } }
I20260808 08:25:11.349541 28329 sys_catalog.cc:458] T 00000000000000000000000000000000 P 293d7890d3b74d37801cec27a4f65bdb [sys.catalog]: This master's current role is: LEADER
I20260808 08:25:11.351904 28352 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:25:11.357864 28352 catalog_manager.cc:1347] Generated new cluster ID: 51838e2ec26b4c77969f103e31af6112
I20260808 08:25:11.357931 28352 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:25:11.370335 28322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:11.365829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38156 (local address 127.22.235.254:41023)
0808 08:25:11.365948 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:11.365952 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:11.366197 (+ 245us) server_negotiation.cc:407] Connection header received
0808 08:25:11.366323 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:11.366326 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:11.366375 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:11.366442 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:11.366966 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:11.367505 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:11.368320 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:11.368557 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:11.368688 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:11.369455 (+ 767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:11.369458 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:11.369460 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:25:11.369900 (+ 440us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:11.369931 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:25:11.369948 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:11.370029 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:11.370128 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 08:25:11.370190 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 08:25:11.386688 28352 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:25:11.387642 28352 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:25:11.393733 28352 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 293d7890d3b74d37801cec27a4f65bdb: Generated new TSK 0
I20260808 08:25:11.394466 28352 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 08:25:11.405395 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 28200
2026-08-08T08:25:11Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1548 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 08:25:11 dist-test-slave-f7th krb5kdc[28373](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 08:25:11 dist-test-slave-f7th krb5kdc[28373](info): set up 2 sockets
Aug 08 08:25:11 dist-test-slave-f7th krb5kdc[28373](info): commencing operation
krb5kdc: starting...
W20260808 08:25:12.562383 23471 mini_kdc.cc:121] Time spent starting KDC: real 1.132s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Aug 08 08:25:12 dist-test-slave-f7th krb5kdc[28373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177512, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T08:25:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:25:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.235.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
I20260808 08:25:12.680141 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:37721
--webserver_interface=127.22.235.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:39225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:37721
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:25:12.789489 28391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:12.789767 28391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:12.789821 28391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:12.793769 28391 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:25:12.793828 28391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:12.793856 28391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:12.793876 28391 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:25:12.793896 28391 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:25:12.799093 28391 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:39225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:37721
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:37721
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.235.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28391
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:12.800333 28391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:12.801441 28391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:12.809860 28405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:12.809926 28401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:12.809897 28400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:12.810235 28391 server_base.cc:1034] running on GCE node
I20260808 08:25:12.810788 28391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:12.811904 28391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:12.813090 28391 hybrid_clock.cc:648] HybridClock initialized: now 1786177512813068 us; error 42 us; skew 500 ppm
Aug 08 08:25:12 dist-test-slave-f7th krb5kdc[28373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177512, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:12.815716 28391 init.cc:377] Logged in from keytab as kudu/127.22.235.254@KRBTEST.COM (short username kudu)
I20260808 08:25:12.816526 28391 webserver.cc:460] Webserver started at http://127.22.235.254:46523/ using document root <none> and password file <none>
I20260808 08:25:12.817073 28391 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:12.817139 28391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:12.817342 28391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:12.819286 28391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "7003f4c44e0046a38e5006bd53151a2f"
format_stamp: "Formatted at 2026-08-08 08:25:12 on dist-test-slave-f7th"
server_key: "f86f0fe1d62178f0f7d1ab7485e4828b"
server_key_iv: "7df8c97e4fe0c10a654ca66e20dbbd00"
server_key_version: "encryptionkey@0"
I20260808 08:25:12.819806 28391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "7003f4c44e0046a38e5006bd53151a2f"
format_stamp: "Formatted at 2026-08-08 08:25:12 on dist-test-slave-f7th"
server_key: "f86f0fe1d62178f0f7d1ab7485e4828b"
server_key_iv: "7df8c97e4fe0c10a654ca66e20dbbd00"
server_key_version: "encryptionkey@0"
I20260808 08:25:12.823310 28391 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260808 08:25:12.825601 28418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:12.826737 28391 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:25:12.826862 28391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "7003f4c44e0046a38e5006bd53151a2f"
format_stamp: "Formatted at 2026-08-08 08:25:12 on dist-test-slave-f7th"
server_key: "f86f0fe1d62178f0f7d1ab7485e4828b"
server_key_iv: "7df8c97e4fe0c10a654ca66e20dbbd00"
server_key_version: "encryptionkey@0"
I20260808 08:25:12.826951 28391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:12.847811 28391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:12.851107 28391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:12.851301 28391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:12.858065 28391 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.254:37721
I20260808 08:25:12.858310 28508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.254:37721 every 8 connection(s)
I20260808 08:25:12.859019 28391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:25:12.861891 28509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:12.865541 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 28391
I20260808 08:25:12.865669 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:25:12.865995 23471 external_mini_cluster.cc:1428] Setting key d24525cbfc0b52daddfb815eafcea8a1
I20260808 08:25:12.867990 28509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7003f4c44e0046a38e5006bd53151a2f: Bootstrap starting.
I20260808 08:25:12.870193 28509 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7003f4c44e0046a38e5006bd53151a2f: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:12.870903 28509 log.cc:824] T 00000000000000000000000000000000 P 7003f4c44e0046a38e5006bd53151a2f: Log is configured to *not* fsync() on all Append() calls
I20260808 08:25:12.872676 28509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7003f4c44e0046a38e5006bd53151a2f: No bootstrap required, opened a new log
Aug 08 08:25:12 dist-test-slave-f7th krb5kdc[28373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177512, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:25:12.876343 28509 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7003f4c44e0046a38e5006bd53151a2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7003f4c44e0046a38e5006bd53151a2f" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 37721 } }
I20260808 08:25:12.876560 28509 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7003f4c44e0046a38e5006bd53151a2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:12.876597 28509 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7003f4c44e0046a38e5006bd53151a2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7003f4c44e0046a38e5006bd53151a2f, State: Initialized, Role: FOLLOWER
I20260808 08:25:12.877044 28509 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7003f4c44e0046a38e5006bd53151a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7003f4c44e0046a38e5006bd53151a2f" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 37721 } }
I20260808 08:25:12.877167 28509 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7003f4c44e0046a38e5006bd53151a2f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:25:12.877228 28509 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7003f4c44e0046a38e5006bd53151a2f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:25:12.877318 28509 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7003f4c44e0046a38e5006bd53151a2f [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:12.878208 28509 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7003f4c44e0046a38e5006bd53151a2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7003f4c44e0046a38e5006bd53151a2f" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 37721 } }
I20260808 08:25:12.878587 28509 leader_election.cc:302] T 00000000000000000000000000000000 P 7003f4c44e0046a38e5006bd53151a2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7003f4c44e0046a38e5006bd53151a2f; no voters:
I20260808 08:25:12.878885 28509 leader_election.cc:288] T 00000000000000000000000000000000 P 7003f4c44e0046a38e5006bd53151a2f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:25:12.878979 28519 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7003f4c44e0046a38e5006bd53151a2f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:25:12.879217 28519 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7003f4c44e0046a38e5006bd53151a2f [term 1 LEADER]: Becoming Leader. State: Replica: 7003f4c44e0046a38e5006bd53151a2f, State: Running, Role: LEADER
I20260808 08:25:12.879576 28519 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7003f4c44e0046a38e5006bd53151a2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7003f4c44e0046a38e5006bd53151a2f" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 37721 } }
I20260808 08:25:12.879631 28515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:12.867424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38186 (local address 127.22.235.254:37721)
0808 08:25:12.867862 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:12.867879 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:12.867915 (+ 36us) server_negotiation.cc:407] Connection header received
0808 08:25:12.868563 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:12.868579 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:12.868944 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:12.869256 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:12.870076 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:12.871110 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:12.871956 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:12.872413 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:12.872558 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:12.875085 (+ 2527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:12.875090 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:12.875102 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:12.875127 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:12.876708 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:12.877084 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:12.877090 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:12.877096 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:12.877233 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:12.877557 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:12.877559 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:12.877560 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:12.877920 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:12.878081 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:12.878373 (+ 292us) server_negotiation.cc:299] Negotiation successful
0808 08:25:12.878653 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":130,"threads_started":1}
I20260808 08:25:12.880097 28509 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7003f4c44e0046a38e5006bd53151a2f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:25:12.881453 28527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7003f4c44e0046a38e5006bd53151a2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7003f4c44e0046a38e5006bd53151a2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7003f4c44e0046a38e5006bd53151a2f" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 37721 } } }
I20260808 08:25:12.881498 28525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7003f4c44e0046a38e5006bd53151a2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7003f4c44e0046a38e5006bd53151a2f. Latest consensus state: current_term: 1 leader_uuid: "7003f4c44e0046a38e5006bd53151a2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7003f4c44e0046a38e5006bd53151a2f" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 37721 } } }
I20260808 08:25:12.881580 28527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7003f4c44e0046a38e5006bd53151a2f [sys.catalog]: This master's current role is: LEADER
I20260808 08:25:12.881596 28525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7003f4c44e0046a38e5006bd53151a2f [sys.catalog]: This master's current role is: LEADER
I20260808 08:25:12.881925 28532 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:25:12.884886 28532 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:25:12.890640 28532 catalog_manager.cc:1347] Generated new cluster ID: fae1f006596243acb929883f725c572b
I20260808 08:25:12.890707 28532 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:25:12.908082 28532 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:25:12.909103 28532 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:25:12.913712 28532 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7003f4c44e0046a38e5006bd53151a2f: Generated new TSK 0
I20260808 08:25:12.914480 28532 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.235.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab.
Entry for principal kudu/127.22.235.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab.
I20260808 08:25:12.941546 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.193:0
--local_ip_for_outbound_sockets=127.22.235.193
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:37721
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab
--principal=kudu/127.22.235.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:39225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:25:13.054800 28554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:13.055060 28554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:13.055120 28554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:13.059072 28554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:13.059128 28554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:13.059202 28554 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.193
I20260808 08:25:13.064280 28554 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:39225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab
--principal=kudu/127.22.235.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:37721
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28554
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.193
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:13.065312 28554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:13.066319 28554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:13.075585 28563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:13.075837 28554 server_base.cc:1034] running on GCE node
W20260808 08:25:13.075562 28561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:13.075591 28567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:13.076311 28554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:13.076960 28554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:13.078130 28554 hybrid_clock.cc:648] HybridClock initialized: now 1786177513078107 us; error 30 us; skew 500 ppm
Aug 08 08:25:13 dist-test-slave-f7th krb5kdc[28373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177513, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:13.080718 28554 init.cc:377] Logged in from keytab as kudu/127.22.235.193@KRBTEST.COM (short username kudu)
I20260808 08:25:13.081586 28554 webserver.cc:460] Webserver started at http://127.22.235.193:34661/ using document root <none> and password file <none>
I20260808 08:25:13.082129 28554 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:13.082194 28554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:13.082407 28554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:13.084396 28554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-0/data/instance:
uuid: "433c64a9053c450dacdc5dcd7cdcb5c7"
format_stamp: "Formatted at 2026-08-08 08:25:13 on dist-test-slave-f7th"
server_key: "661073e1c9f232202e6bb1536824dff0"
server_key_iv: "0301f92523a559fac0383a3fd735e3ac"
server_key_version: "encryptionkey@0"
I20260808 08:25:13.084972 28554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance:
uuid: "433c64a9053c450dacdc5dcd7cdcb5c7"
format_stamp: "Formatted at 2026-08-08 08:25:13 on dist-test-slave-f7th"
server_key: "661073e1c9f232202e6bb1536824dff0"
server_key_iv: "0301f92523a559fac0383a3fd735e3ac"
server_key_version: "encryptionkey@0"
I20260808 08:25:13.088495 28554 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 08:25:13.090780 28579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:13.091907 28554 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:25:13.092033 28554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-0/wal
uuid: "433c64a9053c450dacdc5dcd7cdcb5c7"
format_stamp: "Formatted at 2026-08-08 08:25:13 on dist-test-slave-f7th"
server_key: "661073e1c9f232202e6bb1536824dff0"
server_key_iv: "0301f92523a559fac0383a3fd735e3ac"
server_key_version: "encryptionkey@0"
I20260808 08:25:13.092123 28554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:13.106273 28554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:13.109004 28554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:13.109179 28554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:13.109741 28554 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:25:13.110571 28554 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:25:13.110620 28554 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:13.110671 28554 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:25:13.110698 28554 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:13.119266 28554 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.193:35813
I20260808 08:25:13.119284 28765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.193:35813 every 8 connection(s)
I20260808 08:25:13.120113 28554 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
I20260808 08:25:13.128268 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 28554
I20260808 08:25:13.128397 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance
I20260808 08:25:13.128748 23471 external_mini_cluster.cc:1428] Setting key 4c3a59cbe3d8180a04419b79420ef5da
Aug 08 08:25:13 dist-test-slave-f7th krb5kdc[28373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177513, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:25:13.133324 28515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:13.121840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:58539 (local address 127.22.235.254:37721)
0808 08:25:13.121999 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:13.122004 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:13.122828 (+ 824us) server_negotiation.cc:407] Connection header received
0808 08:25:13.124076 (+ 1248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:13.124079 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:13.124129 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:13.124199 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:13.125351 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:13.125879 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:13.126958 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:13.127201 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:13.127283 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:13.129798 (+ 2515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:13.129801 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:13.129803 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:13.129827 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:13.131270 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:13.131752 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:13.131756 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:13.131758 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:13.131845 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:13.132246 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:13.132249 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:13.132250 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:13.132432 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:13.132523 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:13.133099 (+ 576us) server_negotiation.cc:299] Negotiation successful
0808 08:25:13.133204 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 08:25:13.134367 28772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:13.122248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:37721 (local address 127.22.235.193:58539)
0808 08:25:13.122686 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:13.122726 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:13.123895 (+ 1169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:13.124349 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:13.124356 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:13.124757 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:13.125220 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:13.125233 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:13.125965 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:13.125968 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:13.126844 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:13.126852 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:13.127315 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:13.127318 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:13.127448 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:13.127972 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:13.127987 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:13.129663 (+ 1676us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:13.131379 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:13.131384 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:13.131399 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:13.131656 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:13.131927 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:13.131930 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:13.131932 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:13.132117 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:13.132531 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:13.132537 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:13.132866 (+ 329us) client_negotiation.cc:769] Sending connection context
0808 08:25:13.133073 (+ 207us) client_negotiation.cc:240] Negotiation successful
0808 08:25:13.133402 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":106,"threads_started":1}
I20260808 08:25:13.135584 28767 heartbeater.cc:347] Connected to a master server at 127.22.235.254:37721
I20260808 08:25:13.135821 28767 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:13.136317 28767 heartbeater.cc:511] Master 127.22.235.254:37721 requested a full tablet report, sending...
I20260808 08:25:13.137821 28443 ts_manager.cc:194] Registered new tserver with Master: 433c64a9053c450dacdc5dcd7cdcb5c7 (127.22.235.193:35813)
I20260808 08:25:13.138921 28443 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.193@KRBTEST.COM'} at 127.22.235.193:58539
WARNING: no policy specified for kudu/127.22.235.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab.
Entry for principal kudu/127.22.235.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab.
I20260808 08:25:13.155429 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.194:0
--local_ip_for_outbound_sockets=127.22.235.194
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:37721
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab
--principal=kudu/127.22.235.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:39225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:25:13.269011 28776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:13.269270 28776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:13.269332 28776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:13.273308 28776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:13.273365 28776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:13.273439 28776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.194
I20260808 08:25:13.278563 28776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:39225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab
--principal=kudu/127.22.235.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:37721
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.194
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:13.279556 28776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:13.280578 28776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:13.290038 28792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:13.290038 28788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:13.290059 28787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:13.290325 28776 server_base.cc:1034] running on GCE node
I20260808 08:25:13.290729 28776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:13.291395 28776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:13.292549 28776 hybrid_clock.cc:648] HybridClock initialized: now 1786177513292531 us; error 30 us; skew 500 ppm
Aug 08 08:25:13 dist-test-slave-f7th krb5kdc[28373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177513, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:13.295172 28776 init.cc:377] Logged in from keytab as kudu/127.22.235.194@KRBTEST.COM (short username kudu)
I20260808 08:25:13.296053 28776 webserver.cc:460] Webserver started at http://127.22.235.194:45365/ using document root <none> and password file <none>
I20260808 08:25:13.296591 28776 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:13.296689 28776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:13.296903 28776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:13.298854 28776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-1/data/instance:
uuid: "c328d5619ffe474999dfddb836a88421"
format_stamp: "Formatted at 2026-08-08 08:25:13 on dist-test-slave-f7th"
server_key: "3f47bb11a5977e9c6641941bab0df445"
server_key_iv: "41023fbb7f524cb8fec5b750303ba845"
server_key_version: "encryptionkey@0"
I20260808 08:25:13.299382 28776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance:
uuid: "c328d5619ffe474999dfddb836a88421"
format_stamp: "Formatted at 2026-08-08 08:25:13 on dist-test-slave-f7th"
server_key: "3f47bb11a5977e9c6641941bab0df445"
server_key_iv: "41023fbb7f524cb8fec5b750303ba845"
server_key_version: "encryptionkey@0"
I20260808 08:25:13.302925 28776 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 08:25:13.305310 28804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:13.306455 28776 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:25:13.306569 28776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-1/wal
uuid: "c328d5619ffe474999dfddb836a88421"
format_stamp: "Formatted at 2026-08-08 08:25:13 on dist-test-slave-f7th"
server_key: "3f47bb11a5977e9c6641941bab0df445"
server_key_iv: "41023fbb7f524cb8fec5b750303ba845"
server_key_version: "encryptionkey@0"
I20260808 08:25:13.306663 28776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:13.325121 28776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:13.328112 28776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:13.328280 28776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:13.328871 28776 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:25:13.329687 28776 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:25:13.329734 28776 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:13.329788 28776 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:25:13.329823 28776 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:13.338357 28776 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.194:45053
I20260808 08:25:13.338430 28990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.194:45053 every 8 connection(s)
I20260808 08:25:13.339210 28776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
I20260808 08:25:13.341632 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 28776
I20260808 08:25:13.341751 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance
I20260808 08:25:13.342046 23471 external_mini_cluster.cc:1428] Setting key 156d913b8fbd54b64c6bbe318127de6f
Aug 08 08:25:13 dist-test-slave-f7th krb5kdc[28373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177513, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.194@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.235.195@KRBTEST.COM; defaulting to no policy
I20260808 08:25:13.352119 28515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:13.340928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:38865 (local address 127.22.235.254:37721)
0808 08:25:13.341060 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:13.341064 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:13.341940 (+ 876us) server_negotiation.cc:407] Connection header received
0808 08:25:13.343345 (+ 1405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:13.343348 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:13.343398 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:13.343467 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:13.344554 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:13.345127 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:13.346280 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:13.346501 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:13.346589 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:13.348913 (+ 2324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:13.348916 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:13.348918 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:13.348943 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:13.350208 (+ 1265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:13.350671 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:13.350674 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:13.350675 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:13.350750 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:13.351051 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:13.351055 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:13.351056 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:13.351249 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:13.351340 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:13.351817 (+ 477us) server_negotiation.cc:299] Negotiation successful
0808 08:25:13.351945 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 08:25:13.352989 28993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:13.341308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:37721 (local address 127.22.235.194:38865)
0808 08:25:13.341789 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:13.341837 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:13.343142 (+ 1305us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:13.343631 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:13.343637 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:13.343968 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:13.344415 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:13.344428 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:13.345246 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:13.345249 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:13.346163 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:13.346173 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:13.346633 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:13.346637 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:13.346802 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:13.347327 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:13.347344 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:13.348761 (+ 1417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:13.350312 (+ 1551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:13.350318 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:13.350333 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:13.350580 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:13.350833 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:13.350837 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:13.350839 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:13.350966 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:13.351349 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:13.351355 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:13.351595 (+ 240us) client_negotiation.cc:769] Sending connection context
0808 08:25:13.351760 (+ 165us) client_negotiation.cc:240] Negotiation successful
0808 08:25:13.352071 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":122,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.195@KRBTEST.COM" created.
I20260808 08:25:13.354127 28991 heartbeater.cc:347] Connected to a master server at 127.22.235.254:37721
I20260808 08:25:13.354389 28991 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:13.354974 28991 heartbeater.cc:511] Master 127.22.235.254:37721 requested a full tablet report, sending...
I20260808 08:25:13.356135 28443 ts_manager.cc:194] Registered new tserver with Master: c328d5619ffe474999dfddb836a88421 (127.22.235.194:45053)
I20260808 08:25:13.356818 28443 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.194@KRBTEST.COM'} at 127.22.235.194:38865
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab.
Entry for principal kudu/127.22.235.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab.
I20260808 08:25:13.368399 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.195:0
--local_ip_for_outbound_sockets=127.22.235.195
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:37721
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab
--principal=kudu/127.22.235.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:39225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:25:13.478938 29001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:13.479176 29001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:13.479214 29001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:13.483170 29001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:13.483227 29001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:13.483281 29001 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.195
I20260808 08:25:13.488404 29001 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:39225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab
--principal=kudu/127.22.235.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:37721
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.195
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:13.489396 29001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:13.490327 29001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:13.499576 29008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:13.499568 29009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:13.499650 29001 server_base.cc:1034] running on GCE node
W20260808 08:25:13.499568 29013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:13.500157 29001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:13.500815 29001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:13.501994 29001 hybrid_clock.cc:648] HybridClock initialized: now 1786177513501969 us; error 43 us; skew 500 ppm
Aug 08 08:25:13 dist-test-slave-f7th krb5kdc[28373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177513, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:13.504724 29001 init.cc:377] Logged in from keytab as kudu/127.22.235.195@KRBTEST.COM (short username kudu)
I20260808 08:25:13.505599 29001 webserver.cc:460] Webserver started at http://127.22.235.195:44833/ using document root <none> and password file <none>
I20260808 08:25:13.506143 29001 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:13.506203 29001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:13.506388 29001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:13.508443 29001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-2/data/instance:
uuid: "a9cf04e90072426aa673e513b0af4b8b"
format_stamp: "Formatted at 2026-08-08 08:25:13 on dist-test-slave-f7th"
server_key: "015f06835d860c2828896918d90ed0f0"
server_key_iv: "10d1c7e5df0c639c4788bccef8e42ce3"
server_key_version: "encryptionkey@0"
I20260808 08:25:13.509029 29001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance:
uuid: "a9cf04e90072426aa673e513b0af4b8b"
format_stamp: "Formatted at 2026-08-08 08:25:13 on dist-test-slave-f7th"
server_key: "015f06835d860c2828896918d90ed0f0"
server_key_iv: "10d1c7e5df0c639c4788bccef8e42ce3"
server_key_version: "encryptionkey@0"
I20260808 08:25:13.512784 29001 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 08:25:13.515291 29028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:13.516470 29001 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:25:13.516582 29001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-2/wal
uuid: "a9cf04e90072426aa673e513b0af4b8b"
format_stamp: "Formatted at 2026-08-08 08:25:13 on dist-test-slave-f7th"
server_key: "015f06835d860c2828896918d90ed0f0"
server_key_iv: "10d1c7e5df0c639c4788bccef8e42ce3"
server_key_version: "encryptionkey@0"
I20260808 08:25:13.516705 29001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:13.538220 29001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:13.541227 29001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:13.541375 29001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:13.541905 29001 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:25:13.542685 29001 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:25:13.542729 29001 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:13.542768 29001 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:25:13.542789 29001 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:13.550806 29001 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.195:38365
I20260808 08:25:13.550834 29218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.195:38365 every 8 connection(s)
I20260808 08:25:13.551635 29001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
I20260808 08:25:13.554481 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 29001
I20260808 08:25:13.554595 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance
I20260808 08:25:13.554849 23471 external_mini_cluster.cc:1428] Setting key 2b752ca977ac260202a34332f324fada
Aug 08 08:25:13 dist-test-slave-f7th krb5kdc[28373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177513, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.195@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:25:13.564718 28515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:13.553343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:42507 (local address 127.22.235.254:37721)
0808 08:25:13.553505 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:13.553510 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:13.554328 (+ 818us) server_negotiation.cc:407] Connection header received
0808 08:25:13.555747 (+ 1419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:13.555750 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:13.555807 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:13.555880 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:13.557266 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:13.557786 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:13.558950 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:13.559170 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:13.559256 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:13.561621 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:13.561624 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:13.561626 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:13.561651 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:13.562752 (+ 1101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:13.563224 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:13.563227 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:13.563229 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:13.563307 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:13.563657 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:13.563661 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:13.563663 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:13.563843 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:13.563930 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:13.564405 (+ 475us) server_negotiation.cc:299] Negotiation successful
0808 08:25:13.564532 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 08:25:13.565490 29223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:13.553716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:37721 (local address 127.22.235.195:42507)
0808 08:25:13.554162 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:13.554205 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:13.555537 (+ 1332us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:13.556059 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:13.556066 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:13.556553 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:13.557123 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:13.557137 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:13.557908 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:13.557911 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:13.558806 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:13.558819 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:13.559306 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:13.559310 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:13.559481 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:13.559973 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:13.559990 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:13.561425 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:13.562858 (+ 1433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:13.562864 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:13.562879 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:13.563128 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:13.563389 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:13.563392 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:13.563394 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:13.563565 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:13.563942 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:13.563949 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:13.564179 (+ 230us) client_negotiation.cc:769] Sending connection context
0808 08:25:13.564344 (+ 165us) client_negotiation.cc:240] Negotiation successful
0808 08:25:13.564690 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":111,"threads_started":1}
I20260808 08:25:13.566514 29221 heartbeater.cc:347] Connected to a master server at 127.22.235.254:37721
I20260808 08:25:13.566736 29221 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:13.567261 29221 heartbeater.cc:511] Master 127.22.235.254:37721 requested a full tablet report, sending...
I20260808 08:25:13.568353 28442 ts_manager.cc:194] Registered new tserver with Master: a9cf04e90072426aa673e513b0af4b8b (127.22.235.195:38365)
I20260808 08:25:13.568960 28442 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.195@KRBTEST.COM'} at 127.22.235.195:42507
I20260808 08:25:13.579988 23471 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 08:25:13 dist-test-slave-f7th krb5kdc[28373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177513, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 08 08:25:13 dist-test-slave-f7th krb5kdc[28373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177513, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:25:13.605640 28515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:13.597536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38198 (local address 127.22.235.254:37721)
0808 08:25:13.597692 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:13.597697 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:13.597823 (+ 126us) server_negotiation.cc:407] Connection header received
0808 08:25:13.597969 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:13.597973 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:13.598028 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:13.598095 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:13.598798 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:13.599331 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:13.600144 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:13.600360 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:13.600446 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:13.602700 (+ 2254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:13.602704 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:13.602706 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:13.602733 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:13.604000 (+ 1267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:13.604419 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:13.604422 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:13.604424 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:13.604501 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:13.604782 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:13.604785 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:13.604787 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:13.605017 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:13.605137 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:13.605373 (+ 236us) server_negotiation.cc:299] Negotiation successful
0808 08:25:13.605485 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260808 08:25:13.606045 28442 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:38198
I20260808 08:25:13.606936 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 28554
I20260808 08:25:13.624447 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 28776
I20260808 08:25:13.644245 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 29001
I20260808 08:25:13.652508 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 28391
2026-08-08T08:25:13Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2249 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 08:25:13 dist-test-slave-f7th krb5kdc[29239](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 08:25:13 dist-test-slave-f7th krb5kdc[29239](info): set up 2 sockets
Aug 08 08:25:13 dist-test-slave-f7th krb5kdc[29239](info): commencing operation
krb5kdc: starting...
W20260808 08:25:14.824028 23471 mini_kdc.cc:121] Time spent starting KDC: real 1.144s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Aug 08 08:25:14 dist-test-slave-f7th krb5kdc[29239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177514, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T08:25:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:25:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.235.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
I20260808 08:25:14.947490 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:40313
--webserver_interface=127.22.235.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:45927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:40313
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:25:15.059268 29262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:15.059527 29262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:15.059585 29262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:15.063587 29262 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:25:15.063647 29262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:15.063674 29262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:15.063695 29262 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:25:15.063723 29262 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:25:15.069015 29262 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:45927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:40313
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:40313
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.235.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29262
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:15.070050 29262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:15.071048 29262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:15.079234 29279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:15.079259 29271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:15.079532 29273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:15.079533 29262 server_base.cc:1034] running on GCE node
I20260808 08:25:15.080183 29262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:15.081291 29262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:15.082476 29262 hybrid_clock.cc:648] HybridClock initialized: now 1786177515082454 us; error 32 us; skew 500 ppm
Aug 08 08:25:15 dist-test-slave-f7th krb5kdc[29239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177515, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:15.085263 29262 init.cc:377] Logged in from keytab as kudu/127.22.235.254@KRBTEST.COM (short username kudu)
I20260808 08:25:15.086124 29262 webserver.cc:460] Webserver started at http://127.22.235.254:45311/ using document root <none> and password file <none>
I20260808 08:25:15.086668 29262 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:15.086737 29262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:15.086954 29262 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:15.089043 29262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "3f95b156ace5417facaa0d859860fc9a"
format_stamp: "Formatted at 2026-08-08 08:25:15 on dist-test-slave-f7th"
server_key: "0abe106d1b4652ba14f2020a71b8398e"
server_key_iv: "8490875f509a8a886ca1b831dee3c42b"
server_key_version: "encryptionkey@0"
I20260808 08:25:15.089577 29262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "3f95b156ace5417facaa0d859860fc9a"
format_stamp: "Formatted at 2026-08-08 08:25:15 on dist-test-slave-f7th"
server_key: "0abe106d1b4652ba14f2020a71b8398e"
server_key_iv: "8490875f509a8a886ca1b831dee3c42b"
server_key_version: "encryptionkey@0"
I20260808 08:25:15.093757 29262 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260808 08:25:15.096083 29291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:15.097323 29262 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 08:25:15.097443 29262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "3f95b156ace5417facaa0d859860fc9a"
format_stamp: "Formatted at 2026-08-08 08:25:15 on dist-test-slave-f7th"
server_key: "0abe106d1b4652ba14f2020a71b8398e"
server_key_iv: "8490875f509a8a886ca1b831dee3c42b"
server_key_version: "encryptionkey@0"
I20260808 08:25:15.097540 29262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:15.134794 29262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:15.137746 29262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:15.137944 29262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:15.145144 29262 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.254:40313
I20260808 08:25:15.145183 29366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.254:40313 every 8 connection(s)
I20260808 08:25:15.146122 29262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:25:15.149122 29367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:15.153864 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 29262
I20260808 08:25:15.153986 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:25:15.154243 23471 external_mini_cluster.cc:1428] Setting key 20943a47316c78903ed828205b9213a4
I20260808 08:25:15.154770 29367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f95b156ace5417facaa0d859860fc9a: Bootstrap starting.
I20260808 08:25:15.157164 29367 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f95b156ace5417facaa0d859860fc9a: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:15.158056 29367 log.cc:824] T 00000000000000000000000000000000 P 3f95b156ace5417facaa0d859860fc9a: Log is configured to *not* fsync() on all Append() calls
I20260808 08:25:15.160077 29367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f95b156ace5417facaa0d859860fc9a: No bootstrap required, opened a new log
Aug 08 08:25:15 dist-test-slave-f7th krb5kdc[29239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177514, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:25:15.164485 29367 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f95b156ace5417facaa0d859860fc9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f95b156ace5417facaa0d859860fc9a" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 40313 } }
I20260808 08:25:15.164806 29367 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f95b156ace5417facaa0d859860fc9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:15.164868 29367 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f95b156ace5417facaa0d859860fc9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f95b156ace5417facaa0d859860fc9a, State: Initialized, Role: FOLLOWER
I20260808 08:25:15.165290 29367 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f95b156ace5417facaa0d859860fc9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f95b156ace5417facaa0d859860fc9a" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 40313 } }
I20260808 08:25:15.165414 29367 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f95b156ace5417facaa0d859860fc9a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:25:15.165480 29367 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f95b156ace5417facaa0d859860fc9a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:25:15.165596 29367 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f95b156ace5417facaa0d859860fc9a [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:15.166471 29367 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f95b156ace5417facaa0d859860fc9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f95b156ace5417facaa0d859860fc9a" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 40313 } }
I20260808 08:25:15.166862 29367 leader_election.cc:302] T 00000000000000000000000000000000 P 3f95b156ace5417facaa0d859860fc9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f95b156ace5417facaa0d859860fc9a; no voters:
I20260808 08:25:15.167188 29367 leader_election.cc:288] T 00000000000000000000000000000000 P 3f95b156ace5417facaa0d859860fc9a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:25:15.167290 29378 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f95b156ace5417facaa0d859860fc9a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:25:15.167508 29378 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f95b156ace5417facaa0d859860fc9a [term 1 LEADER]: Becoming Leader. State: Replica: 3f95b156ace5417facaa0d859860fc9a, State: Running, Role: LEADER
I20260808 08:25:15.167871 29378 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f95b156ace5417facaa0d859860fc9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f95b156ace5417facaa0d859860fc9a" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 40313 } }
I20260808 08:25:15.168507 29367 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f95b156ace5417facaa0d859860fc9a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:25:15.169831 29380 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f95b156ace5417facaa0d859860fc9a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f95b156ace5417facaa0d859860fc9a. Latest consensus state: current_term: 1 leader_uuid: "3f95b156ace5417facaa0d859860fc9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f95b156ace5417facaa0d859860fc9a" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 40313 } } }
I20260808 08:25:15.169864 29379 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f95b156ace5417facaa0d859860fc9a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3f95b156ace5417facaa0d859860fc9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f95b156ace5417facaa0d859860fc9a" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 40313 } } }
I20260808 08:25:15.169934 29380 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f95b156ace5417facaa0d859860fc9a [sys.catalog]: This master's current role is: LEADER
I20260808 08:25:15.170001 29379 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f95b156ace5417facaa0d859860fc9a [sys.catalog]: This master's current role is: LEADER
I20260808 08:25:15.169927 29373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:15.155686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41558 (local address 127.22.235.254:40313)
0808 08:25:15.156111 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:15.156127 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:15.156163 (+ 36us) server_negotiation.cc:407] Connection header received
0808 08:25:15.156840 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:15.156860 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:15.157197 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:15.157498 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:15.158370 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:15.159524 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:15.160678 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:15.161428 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:15.161614 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:15.164389 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:15.164410 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:15.164423 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:15.164451 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:15.166244 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:15.166728 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:15.166734 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:15.166740 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:15.166898 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:15.167225 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:15.167228 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:15.167230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:15.167739 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:15.167927 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:15.168597 (+ 670us) server_negotiation.cc:299] Negotiation successful
0808 08:25:15.168926 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":143,"threads_started":1}
W20260808 08:25:15.174068 29406 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3f95b156ace5417facaa0d859860fc9a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 08:25:15.174152 29406 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 08:25:15.174226 29397 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:25:15.175112 29397 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:25:15.180757 29397 catalog_manager.cc:1347] Generated new cluster ID: 1bfe1a02829c4e6ca176cf29b6601029
I20260808 08:25:15.180879 29397 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:25:15.192238 29397 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:25:15.193374 29397 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:25:15.208178 29397 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3f95b156ace5417facaa0d859860fc9a: Generated new TSK 0
I20260808 08:25:15.208920 29397 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.235.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab.
Entry for principal kudu/127.22.235.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab.
I20260808 08:25:15.243378 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.193:0
--local_ip_for_outbound_sockets=127.22.235.193
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:40313
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab
--principal=kudu/127.22.235.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:45927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:25:15.358985 29417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:15.359238 29417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:15.359279 29417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:15.363324 29417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:15.363389 29417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:15.363456 29417 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.193
I20260808 08:25:15.368709 29417 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:45927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab
--principal=kudu/127.22.235.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:40313
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29417
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.193
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:15.369760 29417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:15.370795 29417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:15.380223 29425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:15.380229 29428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:15.380234 29432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:15.380555 29417 server_base.cc:1034] running on GCE node
I20260808 08:25:15.381007 29417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:15.381639 29417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:15.382797 29417 hybrid_clock.cc:648] HybridClock initialized: now 1786177515382782 us; error 26 us; skew 500 ppm
Aug 08 08:25:15 dist-test-slave-f7th krb5kdc[29239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177515, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:15.385423 29417 init.cc:377] Logged in from keytab as kudu/127.22.235.193@KRBTEST.COM (short username kudu)
I20260808 08:25:15.386258 29417 webserver.cc:460] Webserver started at http://127.22.235.193:34655/ using document root <none> and password file <none>
I20260808 08:25:15.386809 29417 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:15.386869 29417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:15.387071 29417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:15.389159 29417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-0/data/instance:
uuid: "9d0e5126068a43d1a6c98901bdaf9825"
format_stamp: "Formatted at 2026-08-08 08:25:15 on dist-test-slave-f7th"
server_key: "624ae122c2d9bb2e389886df0441d04f"
server_key_iv: "bf279021d2bc4f935699a58fbf179a49"
server_key_version: "encryptionkey@0"
I20260808 08:25:15.389798 29417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance:
uuid: "9d0e5126068a43d1a6c98901bdaf9825"
format_stamp: "Formatted at 2026-08-08 08:25:15 on dist-test-slave-f7th"
server_key: "624ae122c2d9bb2e389886df0441d04f"
server_key_iv: "bf279021d2bc4f935699a58fbf179a49"
server_key_version: "encryptionkey@0"
I20260808 08:25:15.393362 29417 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 08:25:15.395610 29444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:15.396787 29417 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 08:25:15.396891 29417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-0/wal
uuid: "9d0e5126068a43d1a6c98901bdaf9825"
format_stamp: "Formatted at 2026-08-08 08:25:15 on dist-test-slave-f7th"
server_key: "624ae122c2d9bb2e389886df0441d04f"
server_key_iv: "bf279021d2bc4f935699a58fbf179a49"
server_key_version: "encryptionkey@0"
I20260808 08:25:15.396978 29417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:15.416678 29417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:15.419535 29417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:15.419724 29417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:15.420349 29417 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:25:15.421301 29417 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:25:15.421348 29417 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:15.421403 29417 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:25:15.421432 29417 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:15.429914 29417 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.193:37917
I20260808 08:25:15.430022 29612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.193:37917 every 8 connection(s)
I20260808 08:25:15.430814 29417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
Aug 08 08:25:15 dist-test-slave-f7th krb5kdc[29239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177515, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:25:15.440318 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 29417
I20260808 08:25:15.440438 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance
I20260808 08:25:15.440828 23471 external_mini_cluster.cc:1428] Setting key 4860cb08e8f3910412b2acf52e6bfa65
I20260808 08:25:15.444608 29373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:15.432706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:53291 (local address 127.22.235.254:40313)
0808 08:25:15.432847 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:15.432850 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:15.433737 (+ 887us) server_negotiation.cc:407] Connection header received
0808 08:25:15.435132 (+ 1395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:15.435135 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:15.435195 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:15.435268 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:15.436476 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:15.437045 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:15.438167 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:15.438398 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:15.438489 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:15.441084 (+ 2595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:15.441087 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:15.441089 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:15.441116 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:15.442560 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:15.443141 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:15.443145 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:15.443147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:15.443248 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:15.443560 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:15.443563 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:15.443565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:15.443779 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:15.443878 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:15.444354 (+ 476us) server_negotiation.cc:299] Negotiation successful
0808 08:25:15.444485 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 08:25:15.445763 29619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:15.433085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:40313 (local address 127.22.235.193:53291)
0808 08:25:15.433575 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:15.433624 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:15.434915 (+ 1291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:15.435457 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:15.435472 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:15.435831 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:15.436327 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:15.436341 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:15.437170 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:15.437173 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:15.438046 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:15.438056 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:15.438551 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:15.438555 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:15.438714 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:15.439263 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:15.439280 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:15.440845 (+ 1565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:15.442690 (+ 1845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:15.442695 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:15.442709 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:15.443024 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:15.443335 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:15.443337 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:15.443338 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:15.443468 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:15.443866 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:15.443870 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:15.444108 (+ 238us) client_negotiation.cc:769] Sending connection context
0808 08:25:15.444304 (+ 196us) client_negotiation.cc:240] Negotiation successful
0808 08:25:15.444704 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":115,"threads_started":1}
I20260808 08:25:15.447068 29613 heartbeater.cc:347] Connected to a master server at 127.22.235.254:40313
I20260808 08:25:15.447305 29613 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:15.447844 29613 heartbeater.cc:511] Master 127.22.235.254:40313 requested a full tablet report, sending...
I20260808 08:25:15.449374 29317 ts_manager.cc:194] Registered new tserver with Master: 9d0e5126068a43d1a6c98901bdaf9825 (127.22.235.193:37917)
I20260808 08:25:15.450505 29317 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.193@KRBTEST.COM'} at 127.22.235.193:53291
WARNING: no policy specified for kudu/127.22.235.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab.
Entry for principal kudu/127.22.235.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab.
I20260808 08:25:15.467782 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.194:0
--local_ip_for_outbound_sockets=127.22.235.194
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:40313
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab
--principal=kudu/127.22.235.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:45927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:25:15.577103 29624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:15.577339 29624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:15.577381 29624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:15.581385 29624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:15.581437 29624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:15.581503 29624 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.194
I20260808 08:25:15.586596 29624 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:45927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab
--principal=kudu/127.22.235.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:40313
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.194
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:15.587589 29624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:15.588546 29624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:15.597710 29633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:15.597858 29624 server_base.cc:1034] running on GCE node
W20260808 08:25:15.597711 29634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:15.597782 29636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:15.598408 29624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:15.599021 29624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:15.600174 29624 hybrid_clock.cc:648] HybridClock initialized: now 1786177515600149 us; error 39 us; skew 500 ppm
Aug 08 08:25:15 dist-test-slave-f7th krb5kdc[29239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177515, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:15.602918 29624 init.cc:377] Logged in from keytab as kudu/127.22.235.194@KRBTEST.COM (short username kudu)
I20260808 08:25:15.603739 29624 webserver.cc:460] Webserver started at http://127.22.235.194:37633/ using document root <none> and password file <none>
I20260808 08:25:15.604277 29624 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:15.604346 29624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:15.604566 29624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:15.606547 29624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-1/data/instance:
uuid: "df17a000819349a18d7b02c756e220ee"
format_stamp: "Formatted at 2026-08-08 08:25:15 on dist-test-slave-f7th"
server_key: "ff4aefd4691ea2137c63d42c03ec3334"
server_key_iv: "4e4b1d6255bf4c9d70193eb1a7beac1c"
server_key_version: "encryptionkey@0"
I20260808 08:25:15.607097 29624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance:
uuid: "df17a000819349a18d7b02c756e220ee"
format_stamp: "Formatted at 2026-08-08 08:25:15 on dist-test-slave-f7th"
server_key: "ff4aefd4691ea2137c63d42c03ec3334"
server_key_iv: "4e4b1d6255bf4c9d70193eb1a7beac1c"
server_key_version: "encryptionkey@0"
I20260808 08:25:15.610754 29624 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 08:25:15.613137 29648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:15.614289 29624 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:25:15.614413 29624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-1/wal
uuid: "df17a000819349a18d7b02c756e220ee"
format_stamp: "Formatted at 2026-08-08 08:25:15 on dist-test-slave-f7th"
server_key: "ff4aefd4691ea2137c63d42c03ec3334"
server_key_iv: "4e4b1d6255bf4c9d70193eb1a7beac1c"
server_key_version: "encryptionkey@0"
I20260808 08:25:15.614516 29624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:15.633523 29624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:15.636219 29624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:15.636402 29624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:15.637046 29624 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:25:15.637878 29624 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:25:15.637925 29624 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:15.637979 29624 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:25:15.638003 29624 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:15.646194 29624 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.194:33183
I20260808 08:25:15.646245 29822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.194:33183 every 8 connection(s)
I20260808 08:25:15.647130 29624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
I20260808 08:25:15.653803 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 29624
I20260808 08:25:15.653926 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance
I20260808 08:25:15.654242 23471 external_mini_cluster.cc:1428] Setting key d560c5fe433488395649fe0629c6191e
Aug 08 08:25:15 dist-test-slave-f7th krb5kdc[29239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177515, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.194@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:25:15.660441 29373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:15.648870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:41329 (local address 127.22.235.254:40313)
0808 08:25:15.649014 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:15.649019 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:15.649855 (+ 836us) server_negotiation.cc:407] Connection header received
0808 08:25:15.651132 (+ 1277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:15.651135 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:15.651182 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:15.651264 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:15.652346 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:15.652932 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:15.654200 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:15.654445 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:15.654526 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:15.657136 (+ 2610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:15.657140 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:15.657142 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:15.657167 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:15.658571 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:15.659031 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:15.659034 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:15.659036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:15.659116 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:15.659414 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:15.659417 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:15.659419 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:15.659602 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:15.659689 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:15.660164 (+ 475us) server_negotiation.cc:299] Negotiation successful
0808 08:25:15.660283 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 08:25:15.661214 29827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:15.649265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:40313 (local address 127.22.235.194:41329)
0808 08:25:15.649709 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:15.649749 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:15.650931 (+ 1182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:15.651407 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:15.651413 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:15.651749 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:15.652222 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:15.652235 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:15.653034 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:15.653037 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:15.654088 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:15.654100 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:15.654592 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:15.654596 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:15.654741 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:15.655336 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:15.655352 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:15.656941 (+ 1589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:15.658682 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:15.658687 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:15.658698 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:15.658928 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:15.659201 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:15.659204 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:15.659206 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:15.659326 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:15.659701 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:15.659708 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:15.659932 (+ 224us) client_negotiation.cc:769] Sending connection context
0808 08:25:15.660101 (+ 169us) client_negotiation.cc:240] Negotiation successful
0808 08:25:15.660400 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":123,"threads_started":1}
I20260808 08:25:15.662271 29824 heartbeater.cc:347] Connected to a master server at 127.22.235.254:40313
I20260808 08:25:15.662541 29824 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:15.663115 29824 heartbeater.cc:511] Master 127.22.235.254:40313 requested a full tablet report, sending...
I20260808 08:25:15.664273 29317 ts_manager.cc:194] Registered new tserver with Master: df17a000819349a18d7b02c756e220ee (127.22.235.194:33183)
WARNING: no policy specified for kudu/127.22.235.195@KRBTEST.COM; defaulting to no policy
I20260808 08:25:15.665059 29317 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.194@KRBTEST.COM'} at 127.22.235.194:41329
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab.
Entry for principal kudu/127.22.235.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab.
I20260808 08:25:15.682977 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.195:0
--local_ip_for_outbound_sockets=127.22.235.195
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:40313
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab
--principal=kudu/127.22.235.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:45927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:25:15.793351 29831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:15.793613 29831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:15.793654 29831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:15.797797 29831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:15.797858 29831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:15.797915 29831 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.195
I20260808 08:25:15.803110 29831 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:45927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab
--principal=kudu/127.22.235.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:40313
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.195
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:15.804070 29831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:15.805101 29831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:15.814554 29845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:15.814565 29846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:15.814563 29848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:15.814941 29831 server_base.cc:1034] running on GCE node
I20260808 08:25:15.815323 29831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:15.815918 29831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:15.817076 29831 hybrid_clock.cc:648] HybridClock initialized: now 1786177515817050 us; error 37 us; skew 500 ppm
Aug 08 08:25:15 dist-test-slave-f7th krb5kdc[29239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177515, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:15.819823 29831 init.cc:377] Logged in from keytab as kudu/127.22.235.195@KRBTEST.COM (short username kudu)
I20260808 08:25:15.820716 29831 webserver.cc:460] Webserver started at http://127.22.235.195:43305/ using document root <none> and password file <none>
I20260808 08:25:15.821250 29831 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:15.821317 29831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:15.821525 29831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:15.823483 29831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-2/data/instance:
uuid: "a67caaa375df4cad8872330d1d35f009"
format_stamp: "Formatted at 2026-08-08 08:25:15 on dist-test-slave-f7th"
server_key: "d16e6e5837db77e09662a969a914d5d8"
server_key_iv: "cce26061c21ddfc4dc666cea7d9505d2"
server_key_version: "encryptionkey@0"
I20260808 08:25:15.824010 29831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance:
uuid: "a67caaa375df4cad8872330d1d35f009"
format_stamp: "Formatted at 2026-08-08 08:25:15 on dist-test-slave-f7th"
server_key: "d16e6e5837db77e09662a969a914d5d8"
server_key_iv: "cce26061c21ddfc4dc666cea7d9505d2"
server_key_version: "encryptionkey@0"
I20260808 08:25:15.827615 29831 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260808 08:25:15.830166 29860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:15.831341 29831 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 08:25:15.831453 29831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-2/wal
uuid: "a67caaa375df4cad8872330d1d35f009"
format_stamp: "Formatted at 2026-08-08 08:25:15 on dist-test-slave-f7th"
server_key: "d16e6e5837db77e09662a969a914d5d8"
server_key_iv: "cce26061c21ddfc4dc666cea7d9505d2"
server_key_version: "encryptionkey@0"
I20260808 08:25:15.831569 29831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:15.848142 29831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:15.850894 29831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:15.851086 29831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:15.851686 29831 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:25:15.852547 29831 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:25:15.852596 29831 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:15.852674 29831 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:25:15.852703 29831 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:15.861101 29831 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.195:33667
I20260808 08:25:15.861130 30049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.195:33667 every 8 connection(s)
I20260808 08:25:15.861963 29831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
I20260808 08:25:15.869072 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 29831
I20260808 08:25:15.869215 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance
I20260808 08:25:15.869481 23471 external_mini_cluster.cc:1428] Setting key fb4444721df15dcabc488343833efff2
Aug 08 08:25:15 dist-test-slave-f7th krb5kdc[29239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177515, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.195@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:25:15.875229 29373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:15.863674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:60631 (local address 127.22.235.254:40313)
0808 08:25:15.863802 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:15.863806 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:15.864633 (+ 827us) server_negotiation.cc:407] Connection header received
0808 08:25:15.865963 (+ 1330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:15.865967 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:15.866037 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:15.866107 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:15.867291 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:15.867809 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:15.869073 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:15.869386 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:15.869480 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:15.872114 (+ 2634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:15.872118 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:15.872121 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:15.872150 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:15.873317 (+ 1167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:15.873785 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:15.873789 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:15.873791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:15.873871 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:15.874195 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:15.874198 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:15.874200 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:15.874380 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:15.874467 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:15.874941 (+ 474us) server_negotiation.cc:299] Negotiation successful
0808 08:25:15.875072 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 08:25:15.875960 30053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:15.864012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:40313 (local address 127.22.235.195:60631)
0808 08:25:15.864453 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:15.864507 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:15.865755 (+ 1248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:15.866308 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:15.866314 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:15.866661 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:15.867145 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:15.867159 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:15.867924 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:15.867927 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:15.868951 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:15.868964 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:15.869551 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:15.869556 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:15.869745 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:15.870303 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:15.870319 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:15.871956 (+ 1637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:15.873423 (+ 1467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:15.873429 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:15.873443 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:15.873690 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:15.873953 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:15.873957 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:15.873959 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:15.874101 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:15.874487 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:15.874494 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:15.874717 (+ 223us) client_negotiation.cc:769] Sending connection context
0808 08:25:15.874879 (+ 162us) client_negotiation.cc:240] Negotiation successful
0808 08:25:15.875184 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":109,"threads_started":1}
I20260808 08:25:15.877013 30050 heartbeater.cc:347] Connected to a master server at 127.22.235.254:40313
I20260808 08:25:15.877233 30050 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:15.877686 30050 heartbeater.cc:511] Master 127.22.235.254:40313 requested a full tablet report, sending...
I20260808 08:25:15.878643 29317 ts_manager.cc:194] Registered new tserver with Master: a67caaa375df4cad8872330d1d35f009 (127.22.235.195:33667)
I20260808 08:25:15.879146 29317 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.195@KRBTEST.COM'} at 127.22.235.195:60631
I20260808 08:25:15.883021 23471 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 08:25:15 dist-test-slave-f7th krb5kdc[29239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 08:25:15 dist-test-slave-f7th krb5kdc[29239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.235.193@KRBTEST.COM
I20260808 08:25:15.907972 30059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:15.899304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43140 (local address 127.22.235.193:37917)
0808 08:25:15.899636 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:15.899652 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:15.899677 (+ 25us) server_negotiation.cc:407] Connection header received
0808 08:25:15.899727 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:15.899732 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:15.899901 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:15.900025 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:15.900753 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:15.901514 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:15.902375 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:15.902738 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:15.902833 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:15.904970 (+ 2137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:15.904979 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:15.904993 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:15.905018 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:15.906293 (+ 1275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:15.906688 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:15.906694 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:15.906695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:15.906775 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:15.907030 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:15.907033 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:15.907035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:15.907265 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:15.907451 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:15.907656 (+ 205us) server_negotiation.cc:299] Negotiation successful
0808 08:25:15.907807 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":115,"threads_started":1}
Aug 08 08:25:15 dist-test-slave-f7th krb5kdc[29239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:25:15.917672 29373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:15.910612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41572 (local address 127.22.235.254:40313)
0808 08:25:15.910747 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:15.910751 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:15.910765 (+ 14us) server_negotiation.cc:407] Connection header received
0808 08:25:15.910875 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:15.910878 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:15.910925 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:15.910987 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:15.911521 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:15.912024 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:15.912772 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:15.913001 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:15.913085 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:15.915078 (+ 1993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:15.915082 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:15.915085 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:15.915109 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:15.916145 (+ 1036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:15.916498 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:15.916502 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:15.916504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:15.916577 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:15.916867 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:15.916871 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:15.916872 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:15.917088 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:15.917203 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:15.917418 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 08:25:15.917525 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260808 08:25:15.918068 29317 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41572
I20260808 08:25:15.918633 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 29417
I20260808 08:25:15.943429 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 29624
I20260808 08:25:15.957631 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 29831
I20260808 08:25:15.969779 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 29262
2026-08-08T08:25:15Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2316 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 08:25:16 dist-test-slave-f7th krb5kdc[30066](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 08:25:16 dist-test-slave-f7th krb5kdc[30066](info): set up 2 sockets
Aug 08 08:25:16 dist-test-slave-f7th krb5kdc[30066](info): commencing operation
krb5kdc: starting...
W20260808 08:25:17.136734 23471 mini_kdc.cc:121] Time spent starting KDC: real 1.140s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Aug 08 08:25:17 dist-test-slave-f7th krb5kdc[30066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T08:25:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:25:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.235.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
I20260808 08:25:17.253611 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:43345
--webserver_interface=127.22.235.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:33309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:43345
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:25:17.363394 30084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:17.363655 30084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:17.363713 30084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:17.367746 30084 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:25:17.367805 30084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:17.367833 30084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:17.367856 30084 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:25:17.367887 30084 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:25:17.373251 30084 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:33309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:43345
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:43345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.235.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:17.374233 30084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:17.375196 30084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:17.383149 30094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:17.383208 30097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:17.383255 30099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:17.383426 30084 server_base.cc:1034] running on GCE node
I20260808 08:25:17.383946 30084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:17.385047 30084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:17.386196 30084 hybrid_clock.cc:648] HybridClock initialized: now 1786177517386172 us; error 37 us; skew 500 ppm
Aug 08 08:25:17 dist-test-slave-f7th krb5kdc[30066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177517, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:17.391845 30084 init.cc:377] Logged in from keytab as kudu/127.22.235.254@KRBTEST.COM (short username kudu)
I20260808 08:25:17.392558 30084 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:17.392656 30084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:17.392869 30084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:17.394830 30084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "e7802ba1223241ef86a555955a233b26"
format_stamp: "Formatted at 2026-08-08 08:25:17 on dist-test-slave-f7th"
server_key: "78c839f3eeb6843a2db5f749e850f9fa"
server_key_iv: "fd4574358f0033babdb3e7e5096f34b3"
server_key_version: "encryptionkey@0"
I20260808 08:25:17.395364 30084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "e7802ba1223241ef86a555955a233b26"
format_stamp: "Formatted at 2026-08-08 08:25:17 on dist-test-slave-f7th"
server_key: "78c839f3eeb6843a2db5f749e850f9fa"
server_key_iv: "fd4574358f0033babdb3e7e5096f34b3"
server_key_version: "encryptionkey@0"
I20260808 08:25:17.398922 30084 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260808 08:25:17.401221 30108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:17.402360 30084 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 08:25:17.402494 30084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "e7802ba1223241ef86a555955a233b26"
format_stamp: "Formatted at 2026-08-08 08:25:17 on dist-test-slave-f7th"
server_key: "78c839f3eeb6843a2db5f749e850f9fa"
server_key_iv: "fd4574358f0033babdb3e7e5096f34b3"
server_key_version: "encryptionkey@0"
I20260808 08:25:17.402585 30084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:17.413661 30084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:17.416194 30084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:17.416363 30084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:17.422991 30084 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.254:43345
I20260808 08:25:17.423220 30199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.254:43345 every 8 connection(s)
I20260808 08:25:17.423615 30084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:25:17.426216 30201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:17.429554 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 30084
I20260808 08:25:17.429666 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:25:17.429901 23471 external_mini_cluster.cc:1428] Setting key 52e213d9c49cae10079fdd63c27ad3d0
I20260808 08:25:17.431795 30201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7802ba1223241ef86a555955a233b26: Bootstrap starting.
I20260808 08:25:17.434218 30201 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e7802ba1223241ef86a555955a233b26: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:17.434916 30201 log.cc:824] T 00000000000000000000000000000000 P e7802ba1223241ef86a555955a233b26: Log is configured to *not* fsync() on all Append() calls
I20260808 08:25:17.436942 30201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7802ba1223241ef86a555955a233b26: No bootstrap required, opened a new log
Aug 08 08:25:17 dist-test-slave-f7th krb5kdc[30066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:25:17.440493 30201 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7802ba1223241ef86a555955a233b26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7802ba1223241ef86a555955a233b26" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 43345 } }
I20260808 08:25:17.440758 30201 raft_consensus.cc:374] T 00000000000000000000000000000000 P e7802ba1223241ef86a555955a233b26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:17.440830 30201 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7802ba1223241ef86a555955a233b26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7802ba1223241ef86a555955a233b26, State: Initialized, Role: FOLLOWER
I20260808 08:25:17.441283 30201 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7802ba1223241ef86a555955a233b26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7802ba1223241ef86a555955a233b26" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 43345 } }
I20260808 08:25:17.441447 30201 raft_consensus.cc:388] T 00000000000000000000000000000000 P e7802ba1223241ef86a555955a233b26 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:25:17.441550 30201 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7802ba1223241ef86a555955a233b26 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:25:17.441648 30201 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7802ba1223241ef86a555955a233b26 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:17.442633 30201 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7802ba1223241ef86a555955a233b26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7802ba1223241ef86a555955a233b26" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 43345 } }
I20260808 08:25:17.442938 30201 leader_election.cc:302] T 00000000000000000000000000000000 P e7802ba1223241ef86a555955a233b26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e7802ba1223241ef86a555955a233b26; no voters:
I20260808 08:25:17.443235 30201 leader_election.cc:288] T 00000000000000000000000000000000 P e7802ba1223241ef86a555955a233b26 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:25:17.443324 30209 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e7802ba1223241ef86a555955a233b26 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:25:17.443485 30209 raft_consensus.cc:686] T 00000000000000000000000000000000 P e7802ba1223241ef86a555955a233b26 [term 1 LEADER]: Becoming Leader. State: Replica: e7802ba1223241ef86a555955a233b26, State: Running, Role: LEADER
I20260808 08:25:17.443603 30207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:17.431257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58978 (local address 127.22.235.254:43345)
0808 08:25:17.431649 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:17.431665 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:17.431700 (+ 35us) server_negotiation.cc:407] Connection header received
0808 08:25:17.432336 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:17.432354 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:17.432729 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:17.433000 (+ 271us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:17.433855 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:17.434864 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:17.435840 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:17.436292 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:17.436407 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:17.438986 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:17.438991 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:17.439004 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:17.439031 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:17.440592 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:17.441080 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:17.441087 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:17.441093 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:17.441225 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:17.441509 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:17.441513 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:17.441515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:17.441961 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:17.442135 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:17.442393 (+ 258us) server_negotiation.cc:299] Negotiation successful
0808 08:25:17.442671 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":105,"threads_started":1}
I20260808 08:25:17.443856 30209 consensus_queue.cc:237] T 00000000000000000000000000000000 P e7802ba1223241ef86a555955a233b26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7802ba1223241ef86a555955a233b26" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 43345 } }
I20260808 08:25:17.444496 30201 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7802ba1223241ef86a555955a233b26 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:25:17.445655 30211 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7802ba1223241ef86a555955a233b26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e7802ba1223241ef86a555955a233b26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7802ba1223241ef86a555955a233b26" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 43345 } } }
I20260808 08:25:17.445663 30213 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7802ba1223241ef86a555955a233b26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7802ba1223241ef86a555955a233b26. Latest consensus state: current_term: 1 leader_uuid: "e7802ba1223241ef86a555955a233b26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7802ba1223241ef86a555955a233b26" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 43345 } } }
I20260808 08:25:17.445787 30211 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7802ba1223241ef86a555955a233b26 [sys.catalog]: This master's current role is: LEADER
I20260808 08:25:17.445789 30213 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7802ba1223241ef86a555955a233b26 [sys.catalog]: This master's current role is: LEADER
W20260808 08:25:17.449165 30233 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e7802ba1223241ef86a555955a233b26: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 08:25:17.449229 30233 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 08:25:17.449285 30222 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:25:17.449988 30222 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:25:17.454746 30222 catalog_manager.cc:1347] Generated new cluster ID: ac62797655ce43cbaaab6dc7729913c6
I20260808 08:25:17.454804 30222 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:25:17.473616 30222 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:25:17.474539 30222 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:25:17.479219 30222 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e7802ba1223241ef86a555955a233b26: Generated new TSK 0
I20260808 08:25:17.479826 30222 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.235.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab.
Entry for principal kudu/127.22.235.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab.
I20260808 08:25:17.507759 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.193:0
--local_ip_for_outbound_sockets=127.22.235.193
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:43345
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab
--principal=kudu/127.22.235.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:33309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:25:17.619391 30243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:17.619632 30243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:17.619673 30243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:17.623553 30243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:17.623613 30243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:17.623672 30243 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.193
I20260808 08:25:17.628728 30243 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:33309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab
--principal=kudu/127.22.235.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:43345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.193
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:17.629684 30243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:17.630635 30243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:17.639762 30254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:17.639739 30252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:17.639784 30251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:17.639986 30243 server_base.cc:1034] running on GCE node
I20260808 08:25:17.640384 30243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:17.641037 30243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:17.642194 30243 hybrid_clock.cc:648] HybridClock initialized: now 1786177517642174 us; error 30 us; skew 500 ppm
Aug 08 08:25:17 dist-test-slave-f7th krb5kdc[30066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177517, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:17.644964 30243 init.cc:377] Logged in from keytab as kudu/127.22.235.193@KRBTEST.COM (short username kudu)
I20260808 08:25:17.645753 30243 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:17.645821 30243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:17.646044 30243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:17.648072 30243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-0/data/instance:
uuid: "b7a5136f262a489b90902828f65926b1"
format_stamp: "Formatted at 2026-08-08 08:25:17 on dist-test-slave-f7th"
server_key: "815e4fddc0a1c981f199cd6d6b6fbdd8"
server_key_iv: "2dc0cfcbc3e775f753ba5539c5a1fedb"
server_key_version: "encryptionkey@0"
I20260808 08:25:17.648663 30243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance:
uuid: "b7a5136f262a489b90902828f65926b1"
format_stamp: "Formatted at 2026-08-08 08:25:17 on dist-test-slave-f7th"
server_key: "815e4fddc0a1c981f199cd6d6b6fbdd8"
server_key_iv: "2dc0cfcbc3e775f753ba5539c5a1fedb"
server_key_version: "encryptionkey@0"
I20260808 08:25:17.652184 30243 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 08:25:17.654570 30277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:17.655736 30243 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 08:25:17.655854 30243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-0/wal
uuid: "b7a5136f262a489b90902828f65926b1"
format_stamp: "Formatted at 2026-08-08 08:25:17 on dist-test-slave-f7th"
server_key: "815e4fddc0a1c981f199cd6d6b6fbdd8"
server_key_iv: "2dc0cfcbc3e775f753ba5539c5a1fedb"
server_key_version: "encryptionkey@0"
I20260808 08:25:17.655942 30243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:17.681704 30243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:17.684357 30243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:17.684531 30243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:17.685130 30243 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:25:17.685966 30243 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:25:17.686017 30243 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:17.686071 30243 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:25:17.686095 30243 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:17.694480 30243 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.193:41085
I20260808 08:25:17.694526 30477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.193:41085 every 8 connection(s)
I20260808 08:25:17.695045 30243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
Aug 08 08:25:17 dist-test-slave-f7th krb5kdc[30066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177517, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:25:17.704704 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 30243
I20260808 08:25:17.704807 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance
I20260808 08:25:17.705080 23471 external_mini_cluster.cc:1428] Setting key ab7465f7ea8be3abdbb3e747414597f2
I20260808 08:25:17.707542 30207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:17.696741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:47801 (local address 127.22.235.254:43345)
0808 08:25:17.696871 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:17.696874 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:17.697986 (+ 1112us) server_negotiation.cc:407] Connection header received
0808 08:25:17.698778 (+ 792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:17.698780 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:17.698825 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:17.698894 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:17.699971 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:17.700499 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:17.701587 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:17.701803 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:17.701875 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:17.704087 (+ 2212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:17.704090 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:17.704092 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:17.704115 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:17.705485 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:17.706027 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:17.706030 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:17.706031 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:17.706110 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:17.706453 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:17.706456 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:17.706458 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:17.706689 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:17.706797 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:17.707257 (+ 460us) server_negotiation.cc:299] Negotiation successful
0808 08:25:17.707379 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 08:25:17.708235 30482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:17.697110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:43345 (local address 127.22.235.193:47801)
0808 08:25:17.697845 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:17.697882 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:17.698615 (+ 733us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:17.699020 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:17.699026 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:17.699347 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:17.699845 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:17.699860 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:17.700599 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:17.700602 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:17.701486 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:17.701506 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:17.701910 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:17.701912 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:17.702040 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:17.702490 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:17.702517 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:17.703938 (+ 1421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:17.705634 (+ 1696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:17.705639 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:17.705650 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:17.705942 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:17.706190 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:17.706193 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:17.706195 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:17.706367 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:17.706813 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:17.706819 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:17.707033 (+ 214us) client_negotiation.cc:769] Sending connection context
0808 08:25:17.707180 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 08:25:17.707424 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":102,"threads_started":1}
I20260808 08:25:17.709424 30480 heartbeater.cc:347] Connected to a master server at 127.22.235.254:43345
I20260808 08:25:17.709647 30480 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:17.710071 30480 heartbeater.cc:511] Master 127.22.235.254:43345 requested a full tablet report, sending...
I20260808 08:25:17.711526 30136 ts_manager.cc:194] Registered new tserver with Master: b7a5136f262a489b90902828f65926b1 (127.22.235.193:41085)
I20260808 08:25:17.712721 30136 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.193@KRBTEST.COM'} at 127.22.235.193:47801
WARNING: no policy specified for kudu/127.22.235.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab.
Entry for principal kudu/127.22.235.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab.
I20260808 08:25:17.731176 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.194:0
--local_ip_for_outbound_sockets=127.22.235.194
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:43345
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab
--principal=kudu/127.22.235.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:33309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:25:17.842164 30489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:17.842419 30489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:17.842463 30489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:17.846537 30489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:17.846598 30489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:17.846657 30489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.194
I20260808 08:25:17.851976 30489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:33309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab
--principal=kudu/127.22.235.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:43345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.194
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:17.853063 30489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:17.854068 30489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:17.863919 30499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:17.863935 30503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:17.863919 30500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:17.864266 30489 server_base.cc:1034] running on GCE node
I20260808 08:25:17.864708 30489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:17.865360 30489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:17.866525 30489 hybrid_clock.cc:648] HybridClock initialized: now 1786177517866499 us; error 36 us; skew 500 ppm
Aug 08 08:25:17 dist-test-slave-f7th krb5kdc[30066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177517, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:17.873052 30489 init.cc:377] Logged in from keytab as kudu/127.22.235.194@KRBTEST.COM (short username kudu)
I20260808 08:25:17.873840 30489 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:17.873912 30489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:17.874107 30489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:17.876107 30489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-1/data/instance:
uuid: "bfc7caa25b474d8199684bc7fa95bd7a"
format_stamp: "Formatted at 2026-08-08 08:25:17 on dist-test-slave-f7th"
server_key: "d05e30e498fcba9681b0e64a9521a424"
server_key_iv: "644c1fd6e7a324cf56e1f3083096692c"
server_key_version: "encryptionkey@0"
I20260808 08:25:17.876675 30489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance:
uuid: "bfc7caa25b474d8199684bc7fa95bd7a"
format_stamp: "Formatted at 2026-08-08 08:25:17 on dist-test-slave-f7th"
server_key: "d05e30e498fcba9681b0e64a9521a424"
server_key_iv: "644c1fd6e7a324cf56e1f3083096692c"
server_key_version: "encryptionkey@0"
I20260808 08:25:17.880226 30489 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 08:25:17.882568 30515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:17.883711 30489 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:25:17.883817 30489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-1/wal
uuid: "bfc7caa25b474d8199684bc7fa95bd7a"
format_stamp: "Formatted at 2026-08-08 08:25:17 on dist-test-slave-f7th"
server_key: "d05e30e498fcba9681b0e64a9521a424"
server_key_iv: "644c1fd6e7a324cf56e1f3083096692c"
server_key_version: "encryptionkey@0"
I20260808 08:25:17.883903 30489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:17.903457 30489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:17.906561 30489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:17.906728 30489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:17.907341 30489 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:25:17.908183 30489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:25:17.908231 30489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:17.908289 30489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:25:17.908314 30489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:17.916836 30489 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.194:41597
I20260808 08:25:17.916857 30703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.194:41597 every 8 connection(s)
I20260808 08:25:17.917423 30489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
Aug 08 08:25:17 dist-test-slave-f7th krb5kdc[30066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177517, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.194@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:25:17.927589 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 30489
I20260808 08:25:17.927727 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance
I20260808 08:25:17.928076 23471 external_mini_cluster.cc:1428] Setting key fa741aceb2d690bcab9acc60bf0b8e0e
I20260808 08:25:17.930658 30207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:17.919090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:60959 (local address 127.22.235.254:43345)
0808 08:25:17.919218 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:17.919221 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:17.920061 (+ 840us) server_negotiation.cc:407] Connection header received
0808 08:25:17.921337 (+ 1276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:17.921341 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:17.921401 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:17.921491 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:17.922642 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:17.923165 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:17.924288 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:17.924523 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:17.924637 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:17.927019 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:17.927023 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:17.927025 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:17.927052 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:17.928548 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:17.929073 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:17.929077 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:17.929079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:17.929172 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:17.929528 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:17.929532 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:17.929533 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:17.929754 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:17.929848 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:17.930350 (+ 502us) server_negotiation.cc:299] Negotiation successful
0808 08:25:17.930482 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 08:25:17.931550 30710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:17.919443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:43345 (local address 127.22.235.194:60959)
0808 08:25:17.919911 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:17.919958 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:17.921130 (+ 1172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:17.921654 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:17.921661 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:17.922022 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:17.922499 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:17.922513 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:17.923284 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:17.923287 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:17.924163 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:17.924173 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:17.924703 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:17.924708 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:17.924875 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:17.925418 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:17.925435 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:17.926847 (+ 1412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:17.928710 (+ 1863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:17.928716 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:17.928732 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:17.928984 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:17.929255 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:17.929257 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:17.929259 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:17.929395 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:17.929859 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:17.929867 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:17.930116 (+ 249us) client_negotiation.cc:769] Sending connection context
0808 08:25:17.930285 (+ 169us) client_negotiation.cc:240] Negotiation successful
0808 08:25:17.930616 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":118,"threads_started":1}
I20260808 08:25:17.932713 30705 heartbeater.cc:347] Connected to a master server at 127.22.235.254:43345
I20260808 08:25:17.932983 30705 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:17.933511 30705 heartbeater.cc:511] Master 127.22.235.254:43345 requested a full tablet report, sending...
I20260808 08:25:17.934621 30136 ts_manager.cc:194] Registered new tserver with Master: bfc7caa25b474d8199684bc7fa95bd7a (127.22.235.194:41597)
I20260808 08:25:17.935179 30136 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.194@KRBTEST.COM'} at 127.22.235.194:60959
WARNING: no policy specified for kudu/127.22.235.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab.
Entry for principal kudu/127.22.235.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab.
I20260808 08:25:17.955157 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.195:0
--local_ip_for_outbound_sockets=127.22.235.195
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:43345
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab
--principal=kudu/127.22.235.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:33309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:25:18.080027 30716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:18.080272 30716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:18.080322 30716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:18.084308 30716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:18.084365 30716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:18.084426 30716 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.195
I20260808 08:25:18.089664 30716 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:33309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab
--principal=kudu/127.22.235.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:43345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.195
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:18.090641 30716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:18.091624 30716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:18.100775 30727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:18.100873 30716 server_base.cc:1034] running on GCE node
W20260808 08:25:18.100780 30725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:18.100796 30729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:18.101363 30716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:18.101977 30716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:18.103137 30716 hybrid_clock.cc:648] HybridClock initialized: now 1786177518103113 us; error 35 us; skew 500 ppm
Aug 08 08:25:18 dist-test-slave-f7th krb5kdc[30066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177518, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:18.105703 30716 init.cc:377] Logged in from keytab as kudu/127.22.235.195@KRBTEST.COM (short username kudu)
I20260808 08:25:18.106437 30716 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:18.106519 30716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:18.106719 30716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:18.108847 30716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-2/data/instance:
uuid: "2cdff96c0d8a4c62a23a448936d9b315"
format_stamp: "Formatted at 2026-08-08 08:25:18 on dist-test-slave-f7th"
server_key: "ab383e33097be84f06e62ea346ed7b8e"
server_key_iv: "11ad691d7b2fb028c27b36145cad9df1"
server_key_version: "encryptionkey@0"
I20260808 08:25:18.109395 30716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance:
uuid: "2cdff96c0d8a4c62a23a448936d9b315"
format_stamp: "Formatted at 2026-08-08 08:25:18 on dist-test-slave-f7th"
server_key: "ab383e33097be84f06e62ea346ed7b8e"
server_key_iv: "11ad691d7b2fb028c27b36145cad9df1"
server_key_version: "encryptionkey@0"
I20260808 08:25:18.112977 30716 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 08:25:18.115317 30740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:18.116514 30716 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:25:18.116637 30716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-2/wal
uuid: "2cdff96c0d8a4c62a23a448936d9b315"
format_stamp: "Formatted at 2026-08-08 08:25:18 on dist-test-slave-f7th"
server_key: "ab383e33097be84f06e62ea346ed7b8e"
server_key_iv: "11ad691d7b2fb028c27b36145cad9df1"
server_key_version: "encryptionkey@0"
I20260808 08:25:18.116734 30716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:18.136828 30716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:18.139480 30716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:18.139652 30716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:18.140218 30716 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:25:18.141093 30716 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:25:18.141140 30716 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:18.141183 30716 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:25:18.141214 30716 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:18.149667 30716 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.195:34477
I20260808 08:25:18.149688 30938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.195:34477 every 8 connection(s)
I20260808 08:25:18.150218 30716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
I20260808 08:25:18.152243 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 30716
I20260808 08:25:18.152354 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance
I20260808 08:25:18.152657 23471 external_mini_cluster.cc:1428] Setting key 811214192351c2652ccc04896cc751a4
Aug 08 08:25:18 dist-test-slave-f7th krb5kdc[30066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177518, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.195@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:25:18.162657 30207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:18.151873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:33045 (local address 127.22.235.254:43345)
0808 08:25:18.152040 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:18.152045 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:18.152946 (+ 901us) server_negotiation.cc:407] Connection header received
0808 08:25:18.154367 (+ 1421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:18.154370 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:18.154416 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:18.154496 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:18.155619 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:18.156134 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:18.157216 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:18.157421 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:18.157508 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:18.159680 (+ 2172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:18.159683 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:18.159685 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:18.159725 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:18.161110 (+ 1385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:18.161517 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:18.161520 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:18.161521 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:18.161590 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:18.161828 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:18.161830 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:18.161831 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:18.161985 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:18.162055 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:18.162436 (+ 381us) server_negotiation.cc:299] Negotiation successful
0808 08:25:18.162537 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 08:25:18.163430 30942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:18.152256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:43345 (local address 127.22.235.195:33045)
0808 08:25:18.152782 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:18.152835 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:18.154145 (+ 1310us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:18.154674 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:18.154683 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:18.155034 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:18.155489 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:18.155512 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:18.156223 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:18.156225 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:18.157126 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:18.157135 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:18.157548 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:18.157551 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:18.157687 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:18.158166 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:18.158182 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:18.159538 (+ 1356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:18.161200 (+ 1662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:18.161204 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:18.161214 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:18.161440 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:18.161647 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:18.161649 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:18.161651 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:18.161766 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:18.162052 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:18.162056 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:18.162265 (+ 209us) client_negotiation.cc:769] Sending connection context
0808 08:25:18.162406 (+ 141us) client_negotiation.cc:240] Negotiation successful
0808 08:25:18.162650 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":134,"threads_started":1}
I20260808 08:25:18.164449 30940 heartbeater.cc:347] Connected to a master server at 127.22.235.254:43345
I20260808 08:25:18.164685 30940 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:18.165160 30940 heartbeater.cc:511] Master 127.22.235.254:43345 requested a full tablet report, sending...
I20260808 08:25:18.166191 30134 ts_manager.cc:194] Registered new tserver with Master: 2cdff96c0d8a4c62a23a448936d9b315 (127.22.235.195:34477)
I20260808 08:25:18.166777 30134 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.195@KRBTEST.COM'} at 127.22.235.195:33045
I20260808 08:25:18.177798 23471 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 08:25:18.186331 30207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:18.179611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58994 (local address 127.22.235.254:43345)
0808 08:25:18.179745 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:18.179750 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:18.179906 (+ 156us) server_negotiation.cc:407] Connection header received
0808 08:25:18.180031 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:18.180034 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:18.180075 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:18.180137 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:18.180888 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:18.181387 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:18.182189 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:18.182397 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:18.182483 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:18.183689 (+ 1206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:18.183692 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:18.183694 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:18.183716 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:18.184756 (+ 1040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:18.185198 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:18.185203 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:18.185204 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:18.185286 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:18.185530 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:18.185533 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:18.185535 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:18.185742 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:18.185862 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:18.186079 (+ 217us) server_negotiation.cc:299] Negotiation successful
0808 08:25:18.186185 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 08:25:18.189023 30134 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58994:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 08:25:18.191879 30134 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 08:25:18.206382 30966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:18.201185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60596 (local address 127.22.235.195:34477)
0808 08:25:18.202098 (+ 913us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:18.202112 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:18.202133 (+ 21us) server_negotiation.cc:407] Connection header received
0808 08:25:18.202200 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:18.202204 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:18.202337 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:18.202476 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:25:18.202712 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:18.203614 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:18.204803 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:18.205642 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:18.205806 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:18.205912 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:18.206124 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 08:25:18.206221 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":809,"thread_start_us":143,"threads_started":1}
I20260808 08:25:18.206382 30962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:18.200111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33648 (local address 127.22.235.194:41597)
0808 08:25:18.200842 (+ 731us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:18.200853 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:18.200949 (+ 96us) server_negotiation.cc:407] Connection header received
0808 08:25:18.201216 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:18.201223 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:18.201435 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:18.201564 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:25:18.202650 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:18.203488 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:18.204887 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:18.205655 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:18.205764 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:18.205885 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:18.206124 (+ 239us) server_negotiation.cc:299] Negotiation successful
0808 08:25:18.206221 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":129,"threads_started":1}
I20260808 08:25:18.206719 30963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:18.200276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48944 (local address 127.22.235.193:41085)
0808 08:25:18.200650 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:18.200657 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:18.201179 (+ 522us) server_negotiation.cc:407] Connection header received
0808 08:25:18.201372 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:18.201377 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:18.201552 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:18.201655 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:25:18.202158 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:18.203086 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:18.204866 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:18.205467 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:18.205545 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:18.205646 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:18.206472 (+ 826us) server_negotiation.cc:299] Negotiation successful
0808 08:25:18.206565 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":62,"threads_started":1}
I20260808 08:25:18.206378 30956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:18.200815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.195:34477 (local address 127.0.0.1:60596)
0808 08:25:18.201616 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:18.201626 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:18.201692 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:18.202462 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:18.202465 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:18.202484 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:25:18.202576 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:18.202583 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:18.203762 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:18.203766 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:18.204696 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:18.204706 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:18.205740 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:18.205743 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:18.205924 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:18.205952 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 08:25:18.206066 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 08:25:18.206209 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":749,"mutex_wait_us":127,"thread_start_us":88,"threads_started":1}
I20260808 08:25:18.207283 30965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:18.200092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:41085 (local address 127.0.0.1:48944)
0808 08:25:18.201084 (+ 992us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:18.201097 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:18.201245 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:18.201830 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:18.201833 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:18.201845 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:25:18.202044 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:18.202052 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:18.203226 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:18.203230 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:18.204718 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:18.204730 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:18.205740 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:18.205743 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:18.206342 (+ 599us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:18.206363 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 08:25:18.206437 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 08:25:18.206509 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":931,"mutex_wait_us":585,"spinlock_wait_cycles":2304,"thread_start_us":102,"threads_started":1}
I20260808 08:25:18.207640 30961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:18.199997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.194:41597 (local address 127.0.0.1:33648)
0808 08:25:18.200812 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:18.200863 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:18.201021 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:18.201799 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:18.201805 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:18.201830 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:25:18.202037 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:18.202046 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:18.203641 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:18.203644 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:18.204785 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:18.204793 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:18.205799 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:18.205802 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:18.205903 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:18.205952 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 08:25:18.206062 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 08:25:18.206209 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"mutex_wait_us":46,"thread_start_us":68,"threads_started":1}
I20260808 08:25:18.209945 30364 tablet_service.cc:1467] Processing CreateTablet for tablet 4f9415e4af004885bcb1e3b7c7c7fb4d (DEFAULT_TABLE table=test-table [id=74e65ffcda0b4187b3e5c9b16bca2ea2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:25:18.210012 30821 tablet_service.cc:1467] Processing CreateTablet for tablet 4f9415e4af004885bcb1e3b7c7c7fb4d (DEFAULT_TABLE table=test-table [id=74e65ffcda0b4187b3e5c9b16bca2ea2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:25:18.210034 30596 tablet_service.cc:1467] Processing CreateTablet for tablet 4f9415e4af004885bcb1e3b7c7c7fb4d (DEFAULT_TABLE table=test-table [id=74e65ffcda0b4187b3e5c9b16bca2ea2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:25:18.210868 30364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f9415e4af004885bcb1e3b7c7c7fb4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:18.211019 30596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f9415e4af004885bcb1e3b7c7c7fb4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:18.211033 30821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f9415e4af004885bcb1e3b7c7c7fb4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:18.217003 30968 tablet_bootstrap.cc:492] T 4f9415e4af004885bcb1e3b7c7c7fb4d P b7a5136f262a489b90902828f65926b1: Bootstrap starting.
I20260808 08:25:18.218216 30971 tablet_bootstrap.cc:492] T 4f9415e4af004885bcb1e3b7c7c7fb4d P 2cdff96c0d8a4c62a23a448936d9b315: Bootstrap starting.
I20260808 08:25:18.218216 30972 tablet_bootstrap.cc:492] T 4f9415e4af004885bcb1e3b7c7c7fb4d P bfc7caa25b474d8199684bc7fa95bd7a: Bootstrap starting.
I20260808 08:25:18.219282 30968 tablet_bootstrap.cc:654] T 4f9415e4af004885bcb1e3b7c7c7fb4d P b7a5136f262a489b90902828f65926b1: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:18.220065 30968 log.cc:824] T 4f9415e4af004885bcb1e3b7c7c7fb4d P b7a5136f262a489b90902828f65926b1: Log is configured to *not* fsync() on all Append() calls
I20260808 08:25:18.220330 30971 tablet_bootstrap.cc:654] T 4f9415e4af004885bcb1e3b7c7c7fb4d P 2cdff96c0d8a4c62a23a448936d9b315: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:18.220330 30972 tablet_bootstrap.cc:654] T 4f9415e4af004885bcb1e3b7c7c7fb4d P bfc7caa25b474d8199684bc7fa95bd7a: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:18.221590 30971 log.cc:824] T 4f9415e4af004885bcb1e3b7c7c7fb4d P 2cdff96c0d8a4c62a23a448936d9b315: Log is configured to *not* fsync() on all Append() calls
I20260808 08:25:18.221590 30972 log.cc:824] T 4f9415e4af004885bcb1e3b7c7c7fb4d P bfc7caa25b474d8199684bc7fa95bd7a: Log is configured to *not* fsync() on all Append() calls
I20260808 08:25:18.222137 30968 tablet_bootstrap.cc:492] T 4f9415e4af004885bcb1e3b7c7c7fb4d P b7a5136f262a489b90902828f65926b1: No bootstrap required, opened a new log
I20260808 08:25:18.222349 30968 ts_tablet_manager.cc:1397] T 4f9415e4af004885bcb1e3b7c7c7fb4d P b7a5136f262a489b90902828f65926b1: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 08:25:18.223446 30971 tablet_bootstrap.cc:492] T 4f9415e4af004885bcb1e3b7c7c7fb4d P 2cdff96c0d8a4c62a23a448936d9b315: No bootstrap required, opened a new log
I20260808 08:25:18.223636 30971 ts_tablet_manager.cc:1397] T 4f9415e4af004885bcb1e3b7c7c7fb4d P 2cdff96c0d8a4c62a23a448936d9b315: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 08:25:18.223817 30972 tablet_bootstrap.cc:492] T 4f9415e4af004885bcb1e3b7c7c7fb4d P bfc7caa25b474d8199684bc7fa95bd7a: No bootstrap required, opened a new log
I20260808 08:25:18.223990 30972 ts_tablet_manager.cc:1397] T 4f9415e4af004885bcb1e3b7c7c7fb4d P bfc7caa25b474d8199684bc7fa95bd7a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 08:25:18.225821 30968 raft_consensus.cc:348] T 4f9415e4af004885bcb1e3b7c7c7fb4d P b7a5136f262a489b90902828f65926b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfc7caa25b474d8199684bc7fa95bd7a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 41597 } } peers { permanent_uuid: "b7a5136f262a489b90902828f65926b1" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41085 } } peers { permanent_uuid: "2cdff96c0d8a4c62a23a448936d9b315" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34477 } }
I20260808 08:25:18.226037 30968 raft_consensus.cc:374] T 4f9415e4af004885bcb1e3b7c7c7fb4d P b7a5136f262a489b90902828f65926b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:18.226099 30968 raft_consensus.cc:729] T 4f9415e4af004885bcb1e3b7c7c7fb4d P b7a5136f262a489b90902828f65926b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7a5136f262a489b90902828f65926b1, State: Initialized, Role: FOLLOWER
I20260808 08:25:18.226639 30968 consensus_queue.cc:260] T 4f9415e4af004885bcb1e3b7c7c7fb4d P b7a5136f262a489b90902828f65926b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfc7caa25b474d8199684bc7fa95bd7a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 41597 } } peers { permanent_uuid: "b7a5136f262a489b90902828f65926b1" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41085 } } peers { permanent_uuid: "2cdff96c0d8a4c62a23a448936d9b315" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34477 } }
I20260808 08:25:18.227387 30971 raft_consensus.cc:348] T 4f9415e4af004885bcb1e3b7c7c7fb4d P 2cdff96c0d8a4c62a23a448936d9b315 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfc7caa25b474d8199684bc7fa95bd7a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 41597 } } peers { permanent_uuid: "b7a5136f262a489b90902828f65926b1" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41085 } } peers { permanent_uuid: "2cdff96c0d8a4c62a23a448936d9b315" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34477 } }
I20260808 08:25:18.227598 30968 ts_tablet_manager.cc:1428] T 4f9415e4af004885bcb1e3b7c7c7fb4d P b7a5136f262a489b90902828f65926b1: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 08:25:18.227645 30971 raft_consensus.cc:374] T 4f9415e4af004885bcb1e3b7c7c7fb4d P 2cdff96c0d8a4c62a23a448936d9b315 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:18.227699 30971 raft_consensus.cc:729] T 4f9415e4af004885bcb1e3b7c7c7fb4d P 2cdff96c0d8a4c62a23a448936d9b315 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cdff96c0d8a4c62a23a448936d9b315, State: Initialized, Role: FOLLOWER
I20260808 08:25:18.227563 30972 raft_consensus.cc:348] T 4f9415e4af004885bcb1e3b7c7c7fb4d P bfc7caa25b474d8199684bc7fa95bd7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfc7caa25b474d8199684bc7fa95bd7a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 41597 } } peers { permanent_uuid: "b7a5136f262a489b90902828f65926b1" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41085 } } peers { permanent_uuid: "2cdff96c0d8a4c62a23a448936d9b315" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34477 } }
I20260808 08:25:18.227789 30972 raft_consensus.cc:374] T 4f9415e4af004885bcb1e3b7c7c7fb4d P bfc7caa25b474d8199684bc7fa95bd7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:18.227836 30972 raft_consensus.cc:729] T 4f9415e4af004885bcb1e3b7c7c7fb4d P bfc7caa25b474d8199684bc7fa95bd7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfc7caa25b474d8199684bc7fa95bd7a, State: Initialized, Role: FOLLOWER
I20260808 08:25:18.227942 30480 heartbeater.cc:503] Master 127.22.235.254:43345 was elected leader, sending a full tablet report...
I20260808 08:25:18.228173 30972 consensus_queue.cc:260] T 4f9415e4af004885bcb1e3b7c7c7fb4d P bfc7caa25b474d8199684bc7fa95bd7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfc7caa25b474d8199684bc7fa95bd7a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 41597 } } peers { permanent_uuid: "b7a5136f262a489b90902828f65926b1" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41085 } } peers { permanent_uuid: "2cdff96c0d8a4c62a23a448936d9b315" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34477 } }
I20260808 08:25:18.228173 30971 consensus_queue.cc:260] T 4f9415e4af004885bcb1e3b7c7c7fb4d P 2cdff96c0d8a4c62a23a448936d9b315 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfc7caa25b474d8199684bc7fa95bd7a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 41597 } } peers { permanent_uuid: "b7a5136f262a489b90902828f65926b1" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41085 } } peers { permanent_uuid: "2cdff96c0d8a4c62a23a448936d9b315" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34477 } }
I20260808 08:25:18.229013 30940 heartbeater.cc:503] Master 127.22.235.254:43345 was elected leader, sending a full tablet report...
I20260808 08:25:18.229054 30705 heartbeater.cc:503] Master 127.22.235.254:43345 was elected leader, sending a full tablet report...
I20260808 08:25:18.229956 30972 ts_tablet_manager.cc:1428] T 4f9415e4af004885bcb1e3b7c7c7fb4d P bfc7caa25b474d8199684bc7fa95bd7a: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 08:25:18.230271 30971 ts_tablet_manager.cc:1428] T 4f9415e4af004885bcb1e3b7c7c7fb4d P 2cdff96c0d8a4c62a23a448936d9b315: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260808 08:25:18.254880 30984 raft_consensus.cc:482] T 4f9415e4af004885bcb1e3b7c7c7fb4d P b7a5136f262a489b90902828f65926b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 08:25:18.255128 30984 raft_consensus.cc:504] T 4f9415e4af004885bcb1e3b7c7c7fb4d P b7a5136f262a489b90902828f65926b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfc7caa25b474d8199684bc7fa95bd7a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 41597 } } peers { permanent_uuid: "b7a5136f262a489b90902828f65926b1" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41085 } } peers { permanent_uuid: "2cdff96c0d8a4c62a23a448936d9b315" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34477 } }
I20260808 08:25:18.256592 30984 leader_election.cc:288] T 4f9415e4af004885bcb1e3b7c7c7fb4d P b7a5136f262a489b90902828f65926b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfc7caa25b474d8199684bc7fa95bd7a (127.22.235.194:41597), 2cdff96c0d8a4c62a23a448936d9b315 (127.22.235.195:34477)
I20260808 08:25:18.260690 30962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:18.257048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:41615 (local address 127.22.235.194:41597)
0808 08:25:18.257201 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:18.257206 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:18.257288 (+ 82us) server_negotiation.cc:407] Connection header received
0808 08:25:18.257466 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:18.257470 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:18.257526 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:18.257603 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:25:18.257911 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:18.258428 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:18.259634 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:18.260145 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:18.260252 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:18.260310 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:18.260476 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 08:25:18.260545 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 08:25:18.260763 30966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:18.257421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:54339 (local address 127.22.235.195:34477)
0808 08:25:18.257549 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:18.257553 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:18.257689 (+ 136us) server_negotiation.cc:407] Connection header received
0808 08:25:18.257823 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:18.257826 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:18.257869 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:18.257930 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:25:18.258218 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:18.258752 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:18.259846 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:18.260236 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:18.260338 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:18.260389 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:18.260565 (+ 176us) server_negotiation.cc:299] Negotiation successful
0808 08:25:18.260651 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 08:25:18.260795 30999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:18.257289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.195:34477 (local address 127.22.235.193:54339)
0808 08:25:18.257613 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:18.257623 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:18.257724 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:18.257948 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:18.257951 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:18.257966 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:25:18.258107 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:18.258113 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:18.258850 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:18.258854 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:18.259732 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:18.259742 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:18.260339 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:18.260343 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:18.260448 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:18.260466 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 08:25:18.260536 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 08:25:18.260604 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":24,"thread_start_us":96,"threads_started":1}
I20260808 08:25:18.260923 30997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:18.256845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.194:41597 (local address 127.22.235.193:41615)
0808 08:25:18.257199 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:18.257214 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:18.257337 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:18.257641 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:18.257644 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:18.257666 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:25:18.257808 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:18.257813 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:18.258535 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:18.258538 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:18.259512 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:18.259522 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:18.260252 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:18.260256 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:18.260356 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:18.260381 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 08:25:18.260449 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 08:25:18.260530 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":26,"spinlock_wait_cycles":29056,"thread_start_us":144,"threads_started":1}
I20260808 08:25:18.261333 30860 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f9415e4af004885bcb1e3b7c7c7fb4d" candidate_uuid: "b7a5136f262a489b90902828f65926b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cdff96c0d8a4c62a23a448936d9b315" is_pre_election: true
I20260808 08:25:18.261548 30627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f9415e4af004885bcb1e3b7c7c7fb4d" candidate_uuid: "b7a5136f262a489b90902828f65926b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfc7caa25b474d8199684bc7fa95bd7a" is_pre_election: true
I20260808 08:25:18.261639 30860 raft_consensus.cc:2454] T 4f9415e4af004885bcb1e3b7c7c7fb4d P 2cdff96c0d8a4c62a23a448936d9b315 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7a5136f262a489b90902828f65926b1 in term 0.
I20260808 08:25:18.261826 30627 raft_consensus.cc:2454] T 4f9415e4af004885bcb1e3b7c7c7fb4d P bfc7caa25b474d8199684bc7fa95bd7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7a5136f262a489b90902828f65926b1 in term 0.
I20260808 08:25:18.262102 30282 leader_election.cc:302] T 4f9415e4af004885bcb1e3b7c7c7fb4d P b7a5136f262a489b90902828f65926b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cdff96c0d8a4c62a23a448936d9b315, b7a5136f262a489b90902828f65926b1; no voters:
I20260808 08:25:18.262343 30984 raft_consensus.cc:2781] T 4f9415e4af004885bcb1e3b7c7c7fb4d P b7a5136f262a489b90902828f65926b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 08:25:18.262430 30984 raft_consensus.cc:482] T 4f9415e4af004885bcb1e3b7c7c7fb4d P b7a5136f262a489b90902828f65926b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 08:25:18.262481 30984 raft_consensus.cc:3037] T 4f9415e4af004885bcb1e3b7c7c7fb4d P b7a5136f262a489b90902828f65926b1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:18.263541 30984 raft_consensus.cc:504] T 4f9415e4af004885bcb1e3b7c7c7fb4d P b7a5136f262a489b90902828f65926b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfc7caa25b474d8199684bc7fa95bd7a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 41597 } } peers { permanent_uuid: "b7a5136f262a489b90902828f65926b1" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41085 } } peers { permanent_uuid: "2cdff96c0d8a4c62a23a448936d9b315" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34477 } }
I20260808 08:25:18.263849 30984 leader_election.cc:288] T 4f9415e4af004885bcb1e3b7c7c7fb4d P b7a5136f262a489b90902828f65926b1 [CANDIDATE]: Term 1 election: Requested vote from peers bfc7caa25b474d8199684bc7fa95bd7a (127.22.235.194:41597), 2cdff96c0d8a4c62a23a448936d9b315 (127.22.235.195:34477)
I20260808 08:25:18.264153 30627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f9415e4af004885bcb1e3b7c7c7fb4d" candidate_uuid: "b7a5136f262a489b90902828f65926b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfc7caa25b474d8199684bc7fa95bd7a"
I20260808 08:25:18.264233 30860 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f9415e4af004885bcb1e3b7c7c7fb4d" candidate_uuid: "b7a5136f262a489b90902828f65926b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cdff96c0d8a4c62a23a448936d9b315"
I20260808 08:25:18.264297 30627 raft_consensus.cc:3037] T 4f9415e4af004885bcb1e3b7c7c7fb4d P bfc7caa25b474d8199684bc7fa95bd7a [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:18.264349 30860 raft_consensus.cc:3037] T 4f9415e4af004885bcb1e3b7c7c7fb4d P 2cdff96c0d8a4c62a23a448936d9b315 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:18.265553 30627 raft_consensus.cc:2454] T 4f9415e4af004885bcb1e3b7c7c7fb4d P bfc7caa25b474d8199684bc7fa95bd7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7a5136f262a489b90902828f65926b1 in term 1.
I20260808 08:25:18.265587 30860 raft_consensus.cc:2454] T 4f9415e4af004885bcb1e3b7c7c7fb4d P 2cdff96c0d8a4c62a23a448936d9b315 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7a5136f262a489b90902828f65926b1 in term 1.
I20260808 08:25:18.265826 30282 leader_election.cc:302] T 4f9415e4af004885bcb1e3b7c7c7fb4d P b7a5136f262a489b90902828f65926b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b7a5136f262a489b90902828f65926b1, bfc7caa25b474d8199684bc7fa95bd7a; no voters:
I20260808 08:25:18.265975 30984 raft_consensus.cc:2781] T 4f9415e4af004885bcb1e3b7c7c7fb4d P b7a5136f262a489b90902828f65926b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:25:18.266196 30984 raft_consensus.cc:686] T 4f9415e4af004885bcb1e3b7c7c7fb4d P b7a5136f262a489b90902828f65926b1 [term 1 LEADER]: Becoming Leader. State: Replica: b7a5136f262a489b90902828f65926b1, State: Running, Role: LEADER
I20260808 08:25:18.266523 30984 consensus_queue.cc:237] T 4f9415e4af004885bcb1e3b7c7c7fb4d P b7a5136f262a489b90902828f65926b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfc7caa25b474d8199684bc7fa95bd7a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 41597 } } peers { permanent_uuid: "b7a5136f262a489b90902828f65926b1" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41085 } } peers { permanent_uuid: "2cdff96c0d8a4c62a23a448936d9b315" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34477 } }
I20260808 08:25:18.269480 30134 catalog_manager.cc:5495] T 4f9415e4af004885bcb1e3b7c7c7fb4d P b7a5136f262a489b90902828f65926b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to b7a5136f262a489b90902828f65926b1 (127.22.235.193). New cstate: current_term: 1 leader_uuid: "b7a5136f262a489b90902828f65926b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfc7caa25b474d8199684bc7fa95bd7a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 41597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7a5136f262a489b90902828f65926b1" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2cdff96c0d8a4c62a23a448936d9b315" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34477 } health_report { overall_health: UNKNOWN } } }
I20260808 08:25:18.288208 30963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:18.284784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48946 (local address 127.22.235.193:41085)
0808 08:25:18.284911 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:18.284916 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:18.284929 (+ 13us) server_negotiation.cc:407] Connection header received
0808 08:25:18.285055 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:18.285058 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:18.285126 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:18.285220 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:25:18.285475 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:18.285982 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:18.286852 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:18.287103 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:18.287201 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:18.287469 (+ 268us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:25:18.287877 (+ 408us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 08:25:18.287991 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:18.288024 (+ 33us) server_negotiation.cc:299] Negotiation successful
0808 08:25:18.288076 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 08:25:18.293874 30860 raft_consensus.cc:1261] T 4f9415e4af004885bcb1e3b7c7c7fb4d P 2cdff96c0d8a4c62a23a448936d9b315 [term 1 FOLLOWER]: Refusing update from remote peer b7a5136f262a489b90902828f65926b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:25:18.293874 30627 raft_consensus.cc:1261] T 4f9415e4af004885bcb1e3b7c7c7fb4d P bfc7caa25b474d8199684bc7fa95bd7a [term 1 FOLLOWER]: Refusing update from remote peer b7a5136f262a489b90902828f65926b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:25:18.294559 30984 consensus_queue.cc:1035] T 4f9415e4af004885bcb1e3b7c7c7fb4d P b7a5136f262a489b90902828f65926b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfc7caa25b474d8199684bc7fa95bd7a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 41597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:25:18.294747 31000 consensus_queue.cc:1035] T 4f9415e4af004885bcb1e3b7c7c7fb4d P b7a5136f262a489b90902828f65926b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cdff96c0d8a4c62a23a448936d9b315" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:25:18.302893 31005 mvcc.cc:205] Tried to move back new op lower bound from 7316183114924625920 to 7316183114822332416. Current Snapshot: MvccSnapshot[applied={T|T < 7316183114924625920}]
I20260808 08:25:18.303071 31006 mvcc.cc:205] Tried to move back new op lower bound from 7316183114924625920 to 7316183114822332416. Current Snapshot: MvccSnapshot[applied={T|T < 7316183114924625920}]
I20260808 08:25:18.310546 30134 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58994:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:25:18.310721 30134 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58994:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:25:18.313134 30134 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e7802ba1223241ef86a555955a233b26: Sending DeleteTablet for 3 replicas of tablet 4f9415e4af004885bcb1e3b7c7c7fb4d
I20260808 08:25:18.313817 30596 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f9415e4af004885bcb1e3b7c7c7fb4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:25:18 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:33648
I20260808 08:25:18.314155 30821 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f9415e4af004885bcb1e3b7c7c7fb4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:25:18 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:60596
I20260808 08:25:18.314594 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 30243
I20260808 08:25:18.314783 31017 tablet_replica.cc:330] T 4f9415e4af004885bcb1e3b7c7c7fb4d P bfc7caa25b474d8199684bc7fa95bd7a: stopping tablet replica
I20260808 08:25:18.315307 31018 tablet_replica.cc:330] T 4f9415e4af004885bcb1e3b7c7c7fb4d P 2cdff96c0d8a4c62a23a448936d9b315: stopping tablet replica
I20260808 08:25:18.315660 31018 raft_consensus.cc:2229] T 4f9415e4af004885bcb1e3b7c7c7fb4d P 2cdff96c0d8a4c62a23a448936d9b315 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 08:25:18.316020 31018 raft_consensus.cc:2258] T 4f9415e4af004885bcb1e3b7c7c7fb4d P 2cdff96c0d8a4c62a23a448936d9b315 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:25:18.316340 31017 raft_consensus.cc:2229] T 4f9415e4af004885bcb1e3b7c7c7fb4d P bfc7caa25b474d8199684bc7fa95bd7a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 08:25:18.316709 31017 raft_consensus.cc:2258] T 4f9415e4af004885bcb1e3b7c7c7fb4d P bfc7caa25b474d8199684bc7fa95bd7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:25:18.318930 31017 ts_tablet_manager.cc:1905] T 4f9415e4af004885bcb1e3b7c7c7fb4d P bfc7caa25b474d8199684bc7fa95bd7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:25:18.322410 31017 ts_tablet_manager.cc:1918] T 4f9415e4af004885bcb1e3b7c7c7fb4d P bfc7caa25b474d8199684bc7fa95bd7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 08:25:18.322556 31017 log.cc:1196] T 4f9415e4af004885bcb1e3b7c7c7fb4d P bfc7caa25b474d8199684bc7fa95bd7a: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-1/wal/wals/4f9415e4af004885bcb1e3b7c7c7fb4d
I20260808 08:25:18.322988 31017 ts_tablet_manager.cc:1939] T 4f9415e4af004885bcb1e3b7c7c7fb4d P bfc7caa25b474d8199684bc7fa95bd7a: Deleting consensus metadata
I20260808 08:25:18.324589 30112 catalog_manager.cc:4841] TS bfc7caa25b474d8199684bc7fa95bd7a (127.22.235.194:41597): tablet 4f9415e4af004885bcb1e3b7c7c7fb4d (table test-table [id=74e65ffcda0b4187b3e5c9b16bca2ea2]) successfully deleted
I20260808 08:25:18.324951 31018 ts_tablet_manager.cc:1905] T 4f9415e4af004885bcb1e3b7c7c7fb4d P 2cdff96c0d8a4c62a23a448936d9b315: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:25:18.327841 31018 ts_tablet_manager.cc:1918] T 4f9415e4af004885bcb1e3b7c7c7fb4d P 2cdff96c0d8a4c62a23a448936d9b315: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 08:25:18.327958 31018 log.cc:1196] T 4f9415e4af004885bcb1e3b7c7c7fb4d P 2cdff96c0d8a4c62a23a448936d9b315: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786177493204855-23471-0/minicluster-data/ts-2/wal/wals/4f9415e4af004885bcb1e3b7c7c7fb4d
I20260808 08:25:18.328307 31018 ts_tablet_manager.cc:1939] T 4f9415e4af004885bcb1e3b7c7c7fb4d P 2cdff96c0d8a4c62a23a448936d9b315: Deleting consensus metadata
I20260808 08:25:18.329825 30112 catalog_manager.cc:4841] TS 2cdff96c0d8a4c62a23a448936d9b315 (127.22.235.195:34477): tablet 4f9415e4af004885bcb1e3b7c7c7fb4d (table test-table [id=74e65ffcda0b4187b3e5c9b16bca2ea2]) successfully deleted
W20260808 08:25:18.335736 30110 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.235.193:41085 (error 108)
I20260808 08:25:18.335843 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 30489
I20260808 08:25:18.340220 30961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:18.339896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:41085 (local address 127.0.0.1:48948)
0808 08:25:18.340052 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:18.340132 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.193:41085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 08:25:18.341058 30110 catalog_manager.cc:4584] TS b7a5136f262a489b90902828f65926b1 (127.22.235.193:41085): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4f9415e4af004885bcb1e3b7c7c7fb4d: Network error: Client connection negotiation failed: client connection to 127.22.235.193:41085: connect: Connection refused (error 111)
I20260808 08:25:18.348067 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 30716
I20260808 08:25:18.358335 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 30084
2026-08-08T08:25:18Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2387 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-08T08:25:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:25:18Z Disabled control of system clock
I20260808 08:25:18.392297 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:44497
--webserver_interface=127.22.235.254
--webserver_port=0
--builtin_ntp_servers=127.22.235.212:41143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:44497
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 08:25:18.500950 31026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:18.501199 31026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:18.501237 31026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:18.505132 31026 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:25:18.505188 31026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:18.505203 31026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:18.505218 31026 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:25:18.505245 31026 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:25:18.509989 31026 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:41143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:44497
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:44497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.235.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:18.511041 31026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:18.512410 31026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:18.520527 31043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:18.520527 31040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:18.520746 31026 server_base.cc:1034] running on GCE node
W20260808 08:25:18.520651 31039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:18.521315 31026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:18.522360 31026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:18.523518 31026 hybrid_clock.cc:648] HybridClock initialized: now 1786177518523498 us; error 33 us; skew 500 ppm
I20260808 08:25:18.525339 31026 webserver.cc:460] Webserver started at http://127.22.235.254:46469/ using document root <none> and password file <none>
I20260808 08:25:18.525902 31026 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:18.525980 31026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:18.526190 31026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:18.528196 31026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "34c3f93de5e54ee8b17d63d6d9088b4f"
format_stamp: "Formatted at 2026-08-08 08:25:18 on dist-test-slave-f7th"
server_key: "866e2ae8b709277884a1f4ed6c9e7075"
server_key_iv: "2d2dd03ddefb6b706b07ab145f4562e0"
server_key_version: "encryptionkey@0"
I20260808 08:25:18.528759 31026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "34c3f93de5e54ee8b17d63d6d9088b4f"
format_stamp: "Formatted at 2026-08-08 08:25:18 on dist-test-slave-f7th"
server_key: "866e2ae8b709277884a1f4ed6c9e7075"
server_key_iv: "2d2dd03ddefb6b706b07ab145f4562e0"
server_key_version: "encryptionkey@0"
I20260808 08:25:18.532254 31026 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 08:25:18.534592 31052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:18.535730 31026 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:25:18.535846 31026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "34c3f93de5e54ee8b17d63d6d9088b4f"
format_stamp: "Formatted at 2026-08-08 08:25:18 on dist-test-slave-f7th"
server_key: "866e2ae8b709277884a1f4ed6c9e7075"
server_key_iv: "2d2dd03ddefb6b706b07ab145f4562e0"
server_key_version: "encryptionkey@0"
I20260808 08:25:18.535940 31026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:18.540370 31026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:18.540984 31026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:18.541141 31026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:18.547511 31026 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.254:44497
I20260808 08:25:18.547523 31133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.254:44497 every 8 connection(s)
I20260808 08:25:18.548388 31026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:25:18.551443 31138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:18.556826 31138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34c3f93de5e54ee8b17d63d6d9088b4f: Bootstrap starting.
I20260808 08:25:18.557510 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 31026
I20260808 08:25:18.557624 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:25:18.557915 23471 external_mini_cluster.cc:1428] Setting key ac4400c29d230d52ae8bdec746b45a5f
I20260808 08:25:18.559675 31138 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34c3f93de5e54ee8b17d63d6d9088b4f: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:18.560356 31138 log.cc:824] T 00000000000000000000000000000000 P 34c3f93de5e54ee8b17d63d6d9088b4f: Log is configured to *not* fsync() on all Append() calls
I20260808 08:25:18.562218 31138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34c3f93de5e54ee8b17d63d6d9088b4f: No bootstrap required, opened a new log
I20260808 08:25:18.563182 31143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:18.559134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45788 (local address 127.22.235.254:44497)
0808 08:25:18.559801 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:18.559817 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:18.559853 (+ 36us) server_negotiation.cc:407] Connection header received
0808 08:25:18.560454 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:18.560491 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:18.560877 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:18.561177 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:18.561926 (+ 749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:18.561931 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:18.561943 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:25:18.561953 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:18.562008 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:25:18.562024 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:18.562120 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:18.562244 (+ 124us) server_negotiation.cc:299] Negotiation successful
0808 08:25:18.562514 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":124,"threads_started":1}
I20260808 08:25:18.565996 31138 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34c3f93de5e54ee8b17d63d6d9088b4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34c3f93de5e54ee8b17d63d6d9088b4f" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 44497 } }
I20260808 08:25:18.566200 31138 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34c3f93de5e54ee8b17d63d6d9088b4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:18.566256 31138 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34c3f93de5e54ee8b17d63d6d9088b4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34c3f93de5e54ee8b17d63d6d9088b4f, State: Initialized, Role: FOLLOWER
I20260808 08:25:18.566641 31138 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34c3f93de5e54ee8b17d63d6d9088b4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34c3f93de5e54ee8b17d63d6d9088b4f" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 44497 } }
I20260808 08:25:18.566766 31138 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34c3f93de5e54ee8b17d63d6d9088b4f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:25:18.566843 31138 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34c3f93de5e54ee8b17d63d6d9088b4f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:25:18.566946 31138 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34c3f93de5e54ee8b17d63d6d9088b4f [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:18.567791 31138 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34c3f93de5e54ee8b17d63d6d9088b4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34c3f93de5e54ee8b17d63d6d9088b4f" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 44497 } }
I20260808 08:25:18.568076 31138 leader_election.cc:302] T 00000000000000000000000000000000 P 34c3f93de5e54ee8b17d63d6d9088b4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34c3f93de5e54ee8b17d63d6d9088b4f; no voters:
I20260808 08:25:18.568308 31138 leader_election.cc:288] T 00000000000000000000000000000000 P 34c3f93de5e54ee8b17d63d6d9088b4f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:25:18.568427 31147 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34c3f93de5e54ee8b17d63d6d9088b4f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:25:18.568693 31147 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34c3f93de5e54ee8b17d63d6d9088b4f [term 1 LEADER]: Becoming Leader. State: Replica: 34c3f93de5e54ee8b17d63d6d9088b4f, State: Running, Role: LEADER
I20260808 08:25:18.568990 31147 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34c3f93de5e54ee8b17d63d6d9088b4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34c3f93de5e54ee8b17d63d6d9088b4f" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 44497 } }
I20260808 08:25:18.569532 31138 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34c3f93de5e54ee8b17d63d6d9088b4f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:25:18.570586 31149 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34c3f93de5e54ee8b17d63d6d9088b4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34c3f93de5e54ee8b17d63d6d9088b4f. Latest consensus state: current_term: 1 leader_uuid: "34c3f93de5e54ee8b17d63d6d9088b4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34c3f93de5e54ee8b17d63d6d9088b4f" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 44497 } } }
I20260808 08:25:18.570621 31148 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34c3f93de5e54ee8b17d63d6d9088b4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "34c3f93de5e54ee8b17d63d6d9088b4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34c3f93de5e54ee8b17d63d6d9088b4f" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 44497 } } }
I20260808 08:25:18.570713 31149 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34c3f93de5e54ee8b17d63d6d9088b4f [sys.catalog]: This master's current role is: LEADER
I20260808 08:25:18.570717 31148 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34c3f93de5e54ee8b17d63d6d9088b4f [sys.catalog]: This master's current role is: LEADER
I20260808 08:25:18.571043 31162 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:25:18.574092 31162 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:25:18.579851 31162 catalog_manager.cc:1347] Generated new cluster ID: d0adec9d0c854f8489b351543b878ef8
I20260808 08:25:18.579906 31162 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:25:18.592979 31162 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:25:18.593291 31162 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:25:18.614641 31162 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34c3f93de5e54ee8b17d63d6d9088b4f: Generated new TSK 0
I20260808 08:25:18.615504 31162 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 08:25:18.622161 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.193:0
--local_ip_for_outbound_sockets=127.22.235.193
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:44497
--builtin_ntp_servers=127.22.235.212:41143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 08:25:18.736088 31180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:18.736336 31180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:18.736377 31180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:18.740504 31180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:18.740558 31180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:18.740650 31180 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.193
I20260808 08:25:18.745383 31180 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:41143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:44497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.193
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:18.746416 31180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:18.747792 31180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:18.757408 31192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:18.757395 31193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:18.757408 31197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:18.757547 31180 server_base.cc:1034] running on GCE node
I20260808 08:25:18.758064 31180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:18.758692 31180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:18.759858 31180 hybrid_clock.cc:648] HybridClock initialized: now 1786177518759836 us; error 39 us; skew 500 ppm
I20260808 08:25:18.761626 31180 webserver.cc:460] Webserver started at http://127.22.235.193:32899/ using document root <none> and password file <none>
I20260808 08:25:18.762182 31180 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:18.762248 31180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:18.762460 31180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:18.764446 31180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-0/data/instance:
uuid: "101d342b95324662ab7b4bebfdab00c2"
format_stamp: "Formatted at 2026-08-08 08:25:18 on dist-test-slave-f7th"
server_key: "34902f1f633c94d3ae478d39a8a68c95"
server_key_iv: "10220b8a491c43549d02814c4d15ea97"
server_key_version: "encryptionkey@0"
I20260808 08:25:18.765017 31180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance:
uuid: "101d342b95324662ab7b4bebfdab00c2"
format_stamp: "Formatted at 2026-08-08 08:25:18 on dist-test-slave-f7th"
server_key: "34902f1f633c94d3ae478d39a8a68c95"
server_key_iv: "10220b8a491c43549d02814c4d15ea97"
server_key_version: "encryptionkey@0"
I20260808 08:25:18.768514 31180 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 08:25:18.770977 31204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:18.772177 31180 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:25:18.772282 31180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-0/wal
uuid: "101d342b95324662ab7b4bebfdab00c2"
format_stamp: "Formatted at 2026-08-08 08:25:18 on dist-test-slave-f7th"
server_key: "34902f1f633c94d3ae478d39a8a68c95"
server_key_iv: "10220b8a491c43549d02814c4d15ea97"
server_key_version: "encryptionkey@0"
I20260808 08:25:18.772379 31180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:18.776844 31180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:18.777402 31180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:18.777552 31180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:18.778060 31180 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:25:18.778944 31180 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:25:18.778997 31180 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:18.779047 31180 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:25:18.779070 31180 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:18.787181 31180 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.193:34643
I20260808 08:25:18.787204 31408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.193:34643 every 8 connection(s)
I20260808 08:25:18.788023 31180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
I20260808 08:25:18.788865 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 31180
I20260808 08:25:18.788965 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance
I20260808 08:25:18.789229 23471 external_mini_cluster.cc:1428] Setting key 1eba05354916bef9846da713828ca6bf
I20260808 08:25:18.791188 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.194:0
--local_ip_for_outbound_sockets=127.22.235.194
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:44497
--builtin_ntp_servers=127.22.235.212:41143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260808 08:25:18.794514 31143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:18.789803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:50235 (local address 127.22.235.254:44497)
0808 08:25:18.789973 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:18.789978 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:18.791151 (+ 1173us) server_negotiation.cc:407] Connection header received
0808 08:25:18.792842 (+ 1691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:18.792847 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:18.792891 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:18.792988 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:18.794075 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:18.794079 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:18.794082 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:25:18.794098 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:18.794126 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:25:18.794136 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:18.794228 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:18.794332 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 08:25:18.794377 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 08:25:18.795305 31417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:18.790239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:44497 (local address 127.22.235.193:50235)
0808 08:25:18.790988 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:18.791042 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:18.792585 (+ 1543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:18.793168 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:18.793174 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:18.793263 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:18.793816 (+ 553us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 08:25:18.793818 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:18.793855 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 08:25:18.793921 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:18.794216 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:18.794221 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:18.794225 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 08:25:18.794342 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 08:25:18.794544 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":136,"threads_started":1}
I20260808 08:25:18.796538 31409 heartbeater.cc:347] Connected to a master server at 127.22.235.254:44497
I20260808 08:25:18.796816 31409 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:18.797312 31409 heartbeater.cc:511] Master 127.22.235.254:44497 requested a full tablet report, sending...
I20260808 08:25:18.798980 31085 ts_manager.cc:194] Registered new tserver with Master: 101d342b95324662ab7b4bebfdab00c2 (127.22.235.193:34643)
W20260808 08:25:18.909395 31419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:18.909641 31419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:18.909684 31419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:18.913727 31419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:18.913776 31419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:18.913846 31419 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.194
I20260808 08:25:18.918533 31419 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:41143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:44497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.194
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:18.919559 31419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:18.920943 31419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:18.930312 31432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:18.930341 31425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:18.930401 31419 server_base.cc:1034] running on GCE node
W20260808 08:25:18.930341 31426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:18.930929 31419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:18.931538 31419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:18.932699 31419 hybrid_clock.cc:648] HybridClock initialized: now 1786177518932673 us; error 36 us; skew 500 ppm
I20260808 08:25:18.934468 31419 webserver.cc:460] Webserver started at http://127.22.235.194:44179/ using document root <none> and password file <none>
I20260808 08:25:18.935045 31419 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:18.935107 31419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:18.935307 31419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:18.937318 31419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-1/data/instance:
uuid: "ac00006170914cafb093d44089e5e06d"
format_stamp: "Formatted at 2026-08-08 08:25:18 on dist-test-slave-f7th"
server_key: "5b84e2d757a4f12266287655b7886415"
server_key_iv: "b615e9d4f0a62bf92416aa222aecc37d"
server_key_version: "encryptionkey@0"
I20260808 08:25:18.937873 31419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance:
uuid: "ac00006170914cafb093d44089e5e06d"
format_stamp: "Formatted at 2026-08-08 08:25:18 on dist-test-slave-f7th"
server_key: "5b84e2d757a4f12266287655b7886415"
server_key_iv: "b615e9d4f0a62bf92416aa222aecc37d"
server_key_version: "encryptionkey@0"
I20260808 08:25:18.941442 31419 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 08:25:18.943718 31442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:18.944913 31419 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:25:18.945034 31419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-1/wal
uuid: "ac00006170914cafb093d44089e5e06d"
format_stamp: "Formatted at 2026-08-08 08:25:18 on dist-test-slave-f7th"
server_key: "5b84e2d757a4f12266287655b7886415"
server_key_iv: "b615e9d4f0a62bf92416aa222aecc37d"
server_key_version: "encryptionkey@0"
I20260808 08:25:18.945127 31419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:18.949501 31419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:18.950074 31419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:18.950223 31419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:18.950722 31419 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:25:18.951576 31419 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:25:18.951622 31419 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:18.951671 31419 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:25:18.951700 31419 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:18.959726 31419 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.194:38781
I20260808 08:25:18.959748 31649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.194:38781 every 8 connection(s)
I20260808 08:25:18.960554 31419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
I20260808 08:25:18.966264 31143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:18.962311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:41673 (local address 127.22.235.254:44497)
0808 08:25:18.962441 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:18.962444 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:18.963415 (+ 971us) server_negotiation.cc:407] Connection header received
0808 08:25:18.964702 (+ 1287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:18.964706 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:18.964753 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:18.964827 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:18.965826 (+ 999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:18.965830 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:18.965833 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:25:18.965846 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:18.965866 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:25:18.965875 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:18.965954 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:18.966081 (+ 127us) server_negotiation.cc:299] Negotiation successful
0808 08:25:18.966129 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 08:25:18.967069 31656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:18.962655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:44497 (local address 127.22.235.194:41673)
0808 08:25:18.963275 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:18.963312 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:18.964473 (+ 1161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:18.964996 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:18.965002 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:18.965088 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:18.965574 (+ 486us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 08:25:18.965576 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:18.965614 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 08:25:18.965680 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:18.965946 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:18.965954 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:18.965960 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 08:25:18.966113 (+ 153us) client_negotiation.cc:240] Negotiation successful
0808 08:25:18.966347 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":110,"threads_started":1}
I20260808 08:25:18.968080 31651 heartbeater.cc:347] Connected to a master server at 127.22.235.254:44497
I20260808 08:25:18.968290 31651 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:18.968393 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 31419
I20260808 08:25:18.968508 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance
I20260808 08:25:18.968745 31651 heartbeater.cc:511] Master 127.22.235.254:44497 requested a full tablet report, sending...
I20260808 08:25:18.968819 23471 external_mini_cluster.cc:1428] Setting key 71aec8fd7d8edb084c025c7f9da24e3f
I20260808 08:25:18.969791 31083 ts_manager.cc:194] Registered new tserver with Master: ac00006170914cafb093d44089e5e06d (127.22.235.194:38781)
I20260808 08:25:18.970726 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.195:0
--local_ip_for_outbound_sockets=127.22.235.195
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:44497
--builtin_ntp_servers=127.22.235.212:41143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 08:25:19.083005 31659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:19.083240 31659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:19.083283 31659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:19.087360 31659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:19.087410 31659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:19.087474 31659 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.195
I20260808 08:25:19.092118 31659 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:41143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:44497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31659
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.195
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:19.093152 31659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:19.094455 31659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:19.103453 31670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:19.103456 31673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:19.103480 31671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:19.103535 31659 server_base.cc:1034] running on GCE node
I20260808 08:25:19.104116 31659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:19.104727 31659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:19.105880 31659 hybrid_clock.cc:648] HybridClock initialized: now 1786177519105860 us; error 32 us; skew 500 ppm
I20260808 08:25:19.107591 31659 webserver.cc:460] Webserver started at http://127.22.235.195:37389/ using document root <none> and password file <none>
I20260808 08:25:19.108167 31659 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:19.108237 31659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:19.108430 31659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:19.110437 31659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-2/data/instance:
uuid: "f03faf52a51442b997fbefd7943ee930"
format_stamp: "Formatted at 2026-08-08 08:25:19 on dist-test-slave-f7th"
server_key: "58671e575953c2b853e2cf3f626877bd"
server_key_iv: "385bb89bc08b0bf26085f239b01ac19b"
server_key_version: "encryptionkey@0"
I20260808 08:25:19.110975 31659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance:
uuid: "f03faf52a51442b997fbefd7943ee930"
format_stamp: "Formatted at 2026-08-08 08:25:19 on dist-test-slave-f7th"
server_key: "58671e575953c2b853e2cf3f626877bd"
server_key_iv: "385bb89bc08b0bf26085f239b01ac19b"
server_key_version: "encryptionkey@0"
I20260808 08:25:19.114553 31659 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 08:25:19.117028 31687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:19.118271 31659 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:25:19.118382 31659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-2/wal
uuid: "f03faf52a51442b997fbefd7943ee930"
format_stamp: "Formatted at 2026-08-08 08:25:19 on dist-test-slave-f7th"
server_key: "58671e575953c2b853e2cf3f626877bd"
server_key_iv: "385bb89bc08b0bf26085f239b01ac19b"
server_key_version: "encryptionkey@0"
I20260808 08:25:19.118477 31659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:19.122856 31659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:19.123420 31659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:19.123570 31659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:19.124087 31659 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:25:19.124989 31659 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:25:19.125037 31659 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:19.125088 31659 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:25:19.125113 31659 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:19.133206 31659 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.195:37793
I20260808 08:25:19.133229 31867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.195:37793 every 8 connection(s)
I20260808 08:25:19.134032 31659 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
I20260808 08:25:19.136940 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 31659
I20260808 08:25:19.137053 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance
I20260808 08:25:19.137300 23471 external_mini_cluster.cc:1428] Setting key 724d347d7379e89279c8e51548425d97
I20260808 08:25:19.140151 31143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:19.135712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:38755 (local address 127.22.235.254:44497)
0808 08:25:19.135844 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:19.135847 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:19.136881 (+ 1034us) server_negotiation.cc:407] Connection header received
0808 08:25:19.138401 (+ 1520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:19.138404 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:19.138455 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:19.138535 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:19.139723 (+ 1188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:19.139727 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:19.139729 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:25:19.139742 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:19.139762 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:25:19.139770 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:19.139840 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:19.139977 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 08:25:19.140021 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 08:25:19.141057 31870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:19.136068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:44497 (local address 127.22.235.195:38755)
0808 08:25:19.136744 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:19.136784 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:19.138188 (+ 1404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:19.138728 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:19.138737 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:19.138855 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:19.139467 (+ 612us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 08:25:19.139469 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:19.139521 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 08:25:19.139590 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:19.139839 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:19.139848 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:19.139853 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 08:25:19.140024 (+ 171us) client_negotiation.cc:240] Negotiation successful
0808 08:25:19.140270 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":111,"threads_started":1}
I20260808 08:25:19.142062 31868 heartbeater.cc:347] Connected to a master server at 127.22.235.254:44497
I20260808 08:25:19.142269 31868 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:19.142663 31868 heartbeater.cc:511] Master 127.22.235.254:44497 requested a full tablet report, sending...
I20260808 08:25:19.143621 31083 ts_manager.cc:194] Registered new tserver with Master: f03faf52a51442b997fbefd7943ee930 (127.22.235.195:37793)
I20260808 08:25:19.151062 23471 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 08:25:19.154398 31143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:19.152865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45800 (local address 127.22.235.254:44497)
0808 08:25:19.152988 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:19.152999 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:19.153206 (+ 207us) server_negotiation.cc:407] Connection header received
0808 08:25:19.153320 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:19.153323 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:19.153360 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:19.153424 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:19.154036 (+ 612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:19.154039 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:19.154042 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:25:19.154053 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:19.154072 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:25:19.154080 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:19.154146 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:19.154236 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 08:25:19.154276 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 08:25:19.157352 31083 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45800:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 08:25:19.160297 31083 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 08:25:19.171953 31894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:19.168595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.194:38781 (local address 127.0.0.1:36640)
0808 08:25:19.169640 (+ 1045us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:19.169652 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:19.169790 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:19.170363 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:19.170370 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:19.170449 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:19.171092 (+ 643us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 08:25:19.171094 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:19.171115 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 08:25:19.171189 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:19.171610 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:19.171621 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:19.171628 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 08:25:19.171744 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 08:25:19.171814 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":982,"thread_start_us":102,"threads_started":1}
I20260808 08:25:19.171967 31890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:19.168900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36640 (local address 127.22.235.194:38781)
0808 08:25:19.169400 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:19.169413 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:19.169733 (+ 320us) server_negotiation.cc:407] Connection header received
0808 08:25:19.169991 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:19.169999 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:19.170183 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:19.170299 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:19.171340 (+ 1041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:19.171347 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:19.171360 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:25:19.171373 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:19.171425 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:25:19.171463 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:19.171565 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:19.171762 (+ 197us) server_negotiation.cc:299] Negotiation successful
0808 08:25:19.171837 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":177,"threads_started":1}
I20260808 08:25:19.172736 31888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:19.168486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.195:37793 (local address 127.0.0.1:50458)
0808 08:25:19.168874 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:19.168925 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:19.169045 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:19.170977 (+ 1932us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:19.170979 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:19.170992 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:19.171470 (+ 478us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 08:25:19.171472 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:19.171482 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 08:25:19.171519 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:19.172507 (+ 988us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:19.172510 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:19.172513 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 08:25:19.172577 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 08:25:19.172600 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":95,"threads_started":1}
I20260808 08:25:19.173030 31895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:19.169852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53096 (local address 127.22.235.193:34643)
0808 08:25:19.170164 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:19.170175 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:19.170860 (+ 685us) server_negotiation.cc:407] Connection header received
0808 08:25:19.170949 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:19.170954 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:19.171141 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:19.171265 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:19.171860 (+ 595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:19.172245 (+ 385us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:19.172257 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:25:19.172272 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:19.172315 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:25:19.172346 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:19.172434 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:19.172805 (+ 371us) server_negotiation.cc:299] Negotiation successful
0808 08:25:19.172879 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":102,"threads_started":1}
I20260808 08:25:19.173346 31889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:19.168666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50458 (local address 127.22.235.195:37793)
0808 08:25:19.169579 (+ 913us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:19.169591 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:19.169618 (+ 27us) server_negotiation.cc:407] Connection header received
0808 08:25:19.169672 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:19.169678 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:19.169875 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:19.170008 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:19.171785 (+ 1777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:19.171790 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:19.171802 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:25:19.171814 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:19.171856 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:25:19.171890 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:19.171985 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:19.173164 (+ 1179us) server_negotiation.cc:299] Negotiation successful
0808 08:25:19.173231 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":795,"thread_start_us":154,"threads_started":1}
I20260808 08:25:19.174856 31535 tablet_service.cc:1467] Processing CreateTablet for tablet 55a50b1b61934f4d8b4b6d79e908a299 (DEFAULT_TABLE table=test-table [id=023488df763140338aaedd7f0a0722cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:25:19.172735 31891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:19.169392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:34643 (local address 127.0.0.1:53096)
0808 08:25:19.170764 (+ 1372us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:19.170776 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:19.170853 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:19.171260 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:19.171263 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:19.171277 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:19.171637 (+ 360us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 08:25:19.171639 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:19.171646 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 08:25:19.171666 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:19.172473 (+ 807us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:19.172476 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:19.172479 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 08:25:19.172548 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 08:25:19.172582 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1293,"thread_start_us":116,"threads_started":1}
I20260808 08:25:19.176031 31535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55a50b1b61934f4d8b4b6d79e908a299. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:19.175663 31765 tablet_service.cc:1467] Processing CreateTablet for tablet 55a50b1b61934f4d8b4b6d79e908a299 (DEFAULT_TABLE table=test-table [id=023488df763140338aaedd7f0a0722cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:25:19.176666 31765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55a50b1b61934f4d8b4b6d79e908a299. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:19.178048 31286 tablet_service.cc:1467] Processing CreateTablet for tablet 55a50b1b61934f4d8b4b6d79e908a299 (DEFAULT_TABLE table=test-table [id=023488df763140338aaedd7f0a0722cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:25:19.179040 31286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55a50b1b61934f4d8b4b6d79e908a299. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:19.182349 31896 tablet_bootstrap.cc:492] T 55a50b1b61934f4d8b4b6d79e908a299 P f03faf52a51442b997fbefd7943ee930: Bootstrap starting.
I20260808 08:25:19.184600 31896 tablet_bootstrap.cc:654] T 55a50b1b61934f4d8b4b6d79e908a299 P f03faf52a51442b997fbefd7943ee930: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:19.185322 31896 log.cc:824] T 55a50b1b61934f4d8b4b6d79e908a299 P f03faf52a51442b997fbefd7943ee930: Log is configured to *not* fsync() on all Append() calls
I20260808 08:25:19.185655 31903 tablet_bootstrap.cc:492] T 55a50b1b61934f4d8b4b6d79e908a299 P 101d342b95324662ab7b4bebfdab00c2: Bootstrap starting.
I20260808 08:25:19.185990 31899 tablet_bootstrap.cc:492] T 55a50b1b61934f4d8b4b6d79e908a299 P ac00006170914cafb093d44089e5e06d: Bootstrap starting.
I20260808 08:25:19.187474 31896 tablet_bootstrap.cc:492] T 55a50b1b61934f4d8b4b6d79e908a299 P f03faf52a51442b997fbefd7943ee930: No bootstrap required, opened a new log
I20260808 08:25:19.187532 31903 tablet_bootstrap.cc:654] T 55a50b1b61934f4d8b4b6d79e908a299 P 101d342b95324662ab7b4bebfdab00c2: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:19.187660 31896 ts_tablet_manager.cc:1397] T 55a50b1b61934f4d8b4b6d79e908a299 P f03faf52a51442b997fbefd7943ee930: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.005s
I20260808 08:25:19.188036 31899 tablet_bootstrap.cc:654] T 55a50b1b61934f4d8b4b6d79e908a299 P ac00006170914cafb093d44089e5e06d: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:19.188134 31903 log.cc:824] T 55a50b1b61934f4d8b4b6d79e908a299 P 101d342b95324662ab7b4bebfdab00c2: Log is configured to *not* fsync() on all Append() calls
I20260808 08:25:19.188840 31899 log.cc:824] T 55a50b1b61934f4d8b4b6d79e908a299 P ac00006170914cafb093d44089e5e06d: Log is configured to *not* fsync() on all Append() calls
I20260808 08:25:19.189879 31903 tablet_bootstrap.cc:492] T 55a50b1b61934f4d8b4b6d79e908a299 P 101d342b95324662ab7b4bebfdab00c2: No bootstrap required, opened a new log
I20260808 08:25:19.190081 31903 ts_tablet_manager.cc:1397] T 55a50b1b61934f4d8b4b6d79e908a299 P 101d342b95324662ab7b4bebfdab00c2: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 08:25:19.190960 31899 tablet_bootstrap.cc:492] T 55a50b1b61934f4d8b4b6d79e908a299 P ac00006170914cafb093d44089e5e06d: No bootstrap required, opened a new log
I20260808 08:25:19.191139 31899 ts_tablet_manager.cc:1397] T 55a50b1b61934f4d8b4b6d79e908a299 P ac00006170914cafb093d44089e5e06d: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260808 08:25:19.191918 31896 raft_consensus.cc:348] T 55a50b1b61934f4d8b4b6d79e908a299 P f03faf52a51442b997fbefd7943ee930 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03faf52a51442b997fbefd7943ee930" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37793 } } peers { permanent_uuid: "ac00006170914cafb093d44089e5e06d" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 38781 } } peers { permanent_uuid: "101d342b95324662ab7b4bebfdab00c2" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 34643 } }
I20260808 08:25:19.192186 31896 raft_consensus.cc:374] T 55a50b1b61934f4d8b4b6d79e908a299 P f03faf52a51442b997fbefd7943ee930 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:19.192242 31896 raft_consensus.cc:729] T 55a50b1b61934f4d8b4b6d79e908a299 P f03faf52a51442b997fbefd7943ee930 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f03faf52a51442b997fbefd7943ee930, State: Initialized, Role: FOLLOWER
I20260808 08:25:19.192749 31896 consensus_queue.cc:260] T 55a50b1b61934f4d8b4b6d79e908a299 P f03faf52a51442b997fbefd7943ee930 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03faf52a51442b997fbefd7943ee930" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37793 } } peers { permanent_uuid: "ac00006170914cafb093d44089e5e06d" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 38781 } } peers { permanent_uuid: "101d342b95324662ab7b4bebfdab00c2" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 34643 } }
I20260808 08:25:19.193434 31868 heartbeater.cc:503] Master 127.22.235.254:44497 was elected leader, sending a full tablet report...
I20260808 08:25:19.193734 31896 ts_tablet_manager.cc:1428] T 55a50b1b61934f4d8b4b6d79e908a299 P f03faf52a51442b997fbefd7943ee930: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 08:25:19.193869 31903 raft_consensus.cc:348] T 55a50b1b61934f4d8b4b6d79e908a299 P 101d342b95324662ab7b4bebfdab00c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03faf52a51442b997fbefd7943ee930" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37793 } } peers { permanent_uuid: "ac00006170914cafb093d44089e5e06d" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 38781 } } peers { permanent_uuid: "101d342b95324662ab7b4bebfdab00c2" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 34643 } }
I20260808 08:25:19.194077 31903 raft_consensus.cc:374] T 55a50b1b61934f4d8b4b6d79e908a299 P 101d342b95324662ab7b4bebfdab00c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:19.194140 31903 raft_consensus.cc:729] T 55a50b1b61934f4d8b4b6d79e908a299 P 101d342b95324662ab7b4bebfdab00c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 101d342b95324662ab7b4bebfdab00c2, State: Initialized, Role: FOLLOWER
I20260808 08:25:19.194518 31903 consensus_queue.cc:260] T 55a50b1b61934f4d8b4b6d79e908a299 P 101d342b95324662ab7b4bebfdab00c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03faf52a51442b997fbefd7943ee930" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37793 } } peers { permanent_uuid: "ac00006170914cafb093d44089e5e06d" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 38781 } } peers { permanent_uuid: "101d342b95324662ab7b4bebfdab00c2" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 34643 } }
I20260808 08:25:19.195109 31409 heartbeater.cc:503] Master 127.22.235.254:44497 was elected leader, sending a full tablet report...
I20260808 08:25:19.195428 31903 ts_tablet_manager.cc:1428] T 55a50b1b61934f4d8b4b6d79e908a299 P 101d342b95324662ab7b4bebfdab00c2: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 08:25:19.196362 31899 raft_consensus.cc:348] T 55a50b1b61934f4d8b4b6d79e908a299 P ac00006170914cafb093d44089e5e06d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03faf52a51442b997fbefd7943ee930" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37793 } } peers { permanent_uuid: "ac00006170914cafb093d44089e5e06d" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 38781 } } peers { permanent_uuid: "101d342b95324662ab7b4bebfdab00c2" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 34643 } }
I20260808 08:25:19.197116 31899 raft_consensus.cc:374] T 55a50b1b61934f4d8b4b6d79e908a299 P ac00006170914cafb093d44089e5e06d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:19.197194 31899 raft_consensus.cc:729] T 55a50b1b61934f4d8b4b6d79e908a299 P ac00006170914cafb093d44089e5e06d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac00006170914cafb093d44089e5e06d, State: Initialized, Role: FOLLOWER
I20260808 08:25:19.197647 31899 consensus_queue.cc:260] T 55a50b1b61934f4d8b4b6d79e908a299 P ac00006170914cafb093d44089e5e06d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03faf52a51442b997fbefd7943ee930" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37793 } } peers { permanent_uuid: "ac00006170914cafb093d44089e5e06d" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 38781 } } peers { permanent_uuid: "101d342b95324662ab7b4bebfdab00c2" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 34643 } }
I20260808 08:25:19.198387 31651 heartbeater.cc:503] Master 127.22.235.254:44497 was elected leader, sending a full tablet report...
I20260808 08:25:19.198573 31899 ts_tablet_manager.cc:1428] T 55a50b1b61934f4d8b4b6d79e908a299 P ac00006170914cafb093d44089e5e06d: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
W20260808 08:25:19.211460 31654 tablet.cc:2369] T 55a50b1b61934f4d8b4b6d79e908a299 P ac00006170914cafb093d44089e5e06d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 08:25:19.290021 31415 tablet.cc:2369] T 55a50b1b61934f4d8b4b6d79e908a299 P 101d342b95324662ab7b4bebfdab00c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 08:25:19.384917 31869 tablet.cc:2369] T 55a50b1b61934f4d8b4b6d79e908a299 P f03faf52a51442b997fbefd7943ee930: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 08:25:19.398371 31913 raft_consensus.cc:482] T 55a50b1b61934f4d8b4b6d79e908a299 P f03faf52a51442b997fbefd7943ee930 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 08:25:19.398625 31913 raft_consensus.cc:504] T 55a50b1b61934f4d8b4b6d79e908a299 P f03faf52a51442b997fbefd7943ee930 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03faf52a51442b997fbefd7943ee930" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37793 } } peers { permanent_uuid: "ac00006170914cafb093d44089e5e06d" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 38781 } } peers { permanent_uuid: "101d342b95324662ab7b4bebfdab00c2" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 34643 } }
I20260808 08:25:19.399946 31913 leader_election.cc:288] T 55a50b1b61934f4d8b4b6d79e908a299 P f03faf52a51442b997fbefd7943ee930 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ac00006170914cafb093d44089e5e06d (127.22.235.194:38781), 101d342b95324662ab7b4bebfdab00c2 (127.22.235.193:34643)
I20260808 08:25:19.401854 31870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:19.400179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.194:38781 (local address 127.22.235.195:45255)
0808 08:25:19.400330 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:19.400345 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:19.400426 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:19.400847 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:19.400851 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:19.400869 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:19.401349 (+ 480us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 08:25:19.401351 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:19.401361 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 08:25:19.401385 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:19.401630 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:19.401633 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:19.401637 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 08:25:19.401696 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 08:25:19.401727 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 08:25:19.401880 31890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:19.400438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:45255 (local address 127.22.235.194:38781)
0808 08:25:19.400599 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:19.400603 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:19.400640 (+ 37us) server_negotiation.cc:407] Connection header received
0808 08:25:19.400685 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:19.400688 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:19.400727 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:19.400805 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:19.401505 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:19.401509 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:19.401511 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:25:19.401523 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:19.401543 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:25:19.401552 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:19.401637 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:19.401700 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 08:25:19.401743 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 08:25:19.401993 31917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:19.400187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:34643 (local address 127.22.235.195:50123)
0808 08:25:19.400716 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:19.400727 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:19.400812 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:19.401039 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:19.401042 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:19.401056 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:19.401512 (+ 456us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 08:25:19.401514 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:19.401523 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 08:25:19.401543 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:19.401779 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:19.401782 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:19.401785 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 08:25:19.401851 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 08:25:19.401881 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"mutex_wait_us":120,"thread_start_us":156,"threads_started":1}
I20260808 08:25:19.401988 31895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:19.400331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:50123 (local address 127.22.235.193:34643)
0808 08:25:19.400464 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:19.400469 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:19.400814 (+ 345us) server_negotiation.cc:407] Connection header received
0808 08:25:19.400887 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:19.400889 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:19.400930 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:19.401011 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:19.401657 (+ 646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:19.401660 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:19.401663 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:25:19.401675 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:19.401694 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:25:19.401704 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:19.401784 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:19.401848 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 08:25:19.401881 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 08:25:19.402520 31323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55a50b1b61934f4d8b4b6d79e908a299" candidate_uuid: "f03faf52a51442b997fbefd7943ee930" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "101d342b95324662ab7b4bebfdab00c2" is_pre_election: true
I20260808 08:25:19.402770 31572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55a50b1b61934f4d8b4b6d79e908a299" candidate_uuid: "f03faf52a51442b997fbefd7943ee930" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac00006170914cafb093d44089e5e06d" is_pre_election: true
I20260808 08:25:19.402796 31323 raft_consensus.cc:2454] T 55a50b1b61934f4d8b4b6d79e908a299 P 101d342b95324662ab7b4bebfdab00c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f03faf52a51442b997fbefd7943ee930 in term 0.
I20260808 08:25:19.403017 31572 raft_consensus.cc:2454] T 55a50b1b61934f4d8b4b6d79e908a299 P ac00006170914cafb093d44089e5e06d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f03faf52a51442b997fbefd7943ee930 in term 0.
I20260808 08:25:19.403213 31694 leader_election.cc:302] T 55a50b1b61934f4d8b4b6d79e908a299 P f03faf52a51442b997fbefd7943ee930 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 101d342b95324662ab7b4bebfdab00c2, f03faf52a51442b997fbefd7943ee930; no voters:
I20260808 08:25:19.403474 31913 raft_consensus.cc:2781] T 55a50b1b61934f4d8b4b6d79e908a299 P f03faf52a51442b997fbefd7943ee930 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 08:25:19.403558 31913 raft_consensus.cc:482] T 55a50b1b61934f4d8b4b6d79e908a299 P f03faf52a51442b997fbefd7943ee930 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 08:25:19.403606 31913 raft_consensus.cc:3037] T 55a50b1b61934f4d8b4b6d79e908a299 P f03faf52a51442b997fbefd7943ee930 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:19.404657 31913 raft_consensus.cc:504] T 55a50b1b61934f4d8b4b6d79e908a299 P f03faf52a51442b997fbefd7943ee930 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03faf52a51442b997fbefd7943ee930" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37793 } } peers { permanent_uuid: "ac00006170914cafb093d44089e5e06d" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 38781 } } peers { permanent_uuid: "101d342b95324662ab7b4bebfdab00c2" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 34643 } }
I20260808 08:25:19.404973 31913 leader_election.cc:288] T 55a50b1b61934f4d8b4b6d79e908a299 P f03faf52a51442b997fbefd7943ee930 [CANDIDATE]: Term 1 election: Requested vote from peers ac00006170914cafb093d44089e5e06d (127.22.235.194:38781), 101d342b95324662ab7b4bebfdab00c2 (127.22.235.193:34643)
I20260808 08:25:19.405287 31572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55a50b1b61934f4d8b4b6d79e908a299" candidate_uuid: "f03faf52a51442b997fbefd7943ee930" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac00006170914cafb093d44089e5e06d"
I20260808 08:25:19.405309 31323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55a50b1b61934f4d8b4b6d79e908a299" candidate_uuid: "f03faf52a51442b997fbefd7943ee930" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "101d342b95324662ab7b4bebfdab00c2"
I20260808 08:25:19.405412 31572 raft_consensus.cc:3037] T 55a50b1b61934f4d8b4b6d79e908a299 P ac00006170914cafb093d44089e5e06d [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:19.405413 31323 raft_consensus.cc:3037] T 55a50b1b61934f4d8b4b6d79e908a299 P 101d342b95324662ab7b4bebfdab00c2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:19.406656 31323 raft_consensus.cc:2454] T 55a50b1b61934f4d8b4b6d79e908a299 P 101d342b95324662ab7b4bebfdab00c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f03faf52a51442b997fbefd7943ee930 in term 1.
I20260808 08:25:19.406656 31572 raft_consensus.cc:2454] T 55a50b1b61934f4d8b4b6d79e908a299 P ac00006170914cafb093d44089e5e06d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f03faf52a51442b997fbefd7943ee930 in term 1.
I20260808 08:25:19.406920 31694 leader_election.cc:302] T 55a50b1b61934f4d8b4b6d79e908a299 P f03faf52a51442b997fbefd7943ee930 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 101d342b95324662ab7b4bebfdab00c2, f03faf52a51442b997fbefd7943ee930; no voters:
I20260808 08:25:19.407069 31913 raft_consensus.cc:2781] T 55a50b1b61934f4d8b4b6d79e908a299 P f03faf52a51442b997fbefd7943ee930 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:25:19.407280 31913 raft_consensus.cc:686] T 55a50b1b61934f4d8b4b6d79e908a299 P f03faf52a51442b997fbefd7943ee930 [term 1 LEADER]: Becoming Leader. State: Replica: f03faf52a51442b997fbefd7943ee930, State: Running, Role: LEADER
I20260808 08:25:19.407614 31913 consensus_queue.cc:237] T 55a50b1b61934f4d8b4b6d79e908a299 P f03faf52a51442b997fbefd7943ee930 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03faf52a51442b997fbefd7943ee930" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37793 } } peers { permanent_uuid: "ac00006170914cafb093d44089e5e06d" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 38781 } } peers { permanent_uuid: "101d342b95324662ab7b4bebfdab00c2" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 34643 } }
I20260808 08:25:19.410804 31085 catalog_manager.cc:5495] T 55a50b1b61934f4d8b4b6d79e908a299 P f03faf52a51442b997fbefd7943ee930 reported cstate change: term changed from 0 to 1, leader changed from <none> to f03faf52a51442b997fbefd7943ee930 (127.22.235.195). New cstate: current_term: 1 leader_uuid: "f03faf52a51442b997fbefd7943ee930" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03faf52a51442b997fbefd7943ee930" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37793 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac00006170914cafb093d44089e5e06d" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 38781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "101d342b95324662ab7b4bebfdab00c2" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 34643 } health_report { overall_health: UNKNOWN } } }
I20260808 08:25:19.461763 31889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:19.460514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50464 (local address 127.22.235.195:37793)
0808 08:25:19.460672 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:19.460676 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:19.460689 (+ 13us) server_negotiation.cc:407] Connection header received
0808 08:25:19.460766 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:19.460768 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:19.460829 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:19.460896 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:19.461458 (+ 562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:19.461461 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:19.461464 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:25:19.461474 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:19.461504 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:25:19.461513 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:19.461580 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:19.461625 (+ 45us) server_negotiation.cc:299] Negotiation successful
0808 08:25:19.461658 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 08:25:19.467492 31323 raft_consensus.cc:1261] T 55a50b1b61934f4d8b4b6d79e908a299 P 101d342b95324662ab7b4bebfdab00c2 [term 1 FOLLOWER]: Refusing update from remote peer f03faf52a51442b997fbefd7943ee930: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:25:19.467489 31572 raft_consensus.cc:1261] T 55a50b1b61934f4d8b4b6d79e908a299 P ac00006170914cafb093d44089e5e06d [term 1 FOLLOWER]: Refusing update from remote peer f03faf52a51442b997fbefd7943ee930: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:25:19.468187 31918 consensus_queue.cc:1035] T 55a50b1b61934f4d8b4b6d79e908a299 P f03faf52a51442b997fbefd7943ee930 [LEADER]: Connected to new peer: Peer: permanent_uuid: "101d342b95324662ab7b4bebfdab00c2" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 34643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:25:19.468350 31913 consensus_queue.cc:1035] T 55a50b1b61934f4d8b4b6d79e908a299 P f03faf52a51442b997fbefd7943ee930 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac00006170914cafb093d44089e5e06d" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 38781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:25:19.475574 31925 mvcc.cc:205] Tried to move back new op lower bound from 7316183119731933184 to 7316183119495962624. Current Snapshot: MvccSnapshot[applied={T|T < 7316183119731933184}]
I20260808 08:25:19.479056 31924 mvcc.cc:205] Tried to move back new op lower bound from 7316183119731933184 to 7316183119495962624. Current Snapshot: MvccSnapshot[applied={T|T < 7316183119731933184 or (T in {7316183119731933184})}]
I20260808 08:25:19.483527 31083 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:25:19.483668 31083 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:25:19.485930 31083 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 34c3f93de5e54ee8b17d63d6d9088b4f: Sending DeleteTablet for 3 replicas of tablet 55a50b1b61934f4d8b4b6d79e908a299
I20260808 08:25:19.486670 31535 tablet_service.cc:1514] Processing DeleteTablet for tablet 55a50b1b61934f4d8b4b6d79e908a299 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:25:19 UTC) from {username='slave'} at 127.0.0.1:36640
I20260808 08:25:19.486651 31765 tablet_service.cc:1514] Processing DeleteTablet for tablet 55a50b1b61934f4d8b4b6d79e908a299 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:25:19 UTC) from {username='slave'} at 127.0.0.1:50458
I20260808 08:25:19.487411 31935 tablet_replica.cc:330] T 55a50b1b61934f4d8b4b6d79e908a299 P f03faf52a51442b997fbefd7943ee930: stopping tablet replica
I20260808 08:25:19.487393 31286 tablet_service.cc:1514] Processing DeleteTablet for tablet 55a50b1b61934f4d8b4b6d79e908a299 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:25:19 UTC) from {username='slave'} at 127.0.0.1:53096
I20260808 08:25:19.487718 31935 raft_consensus.cc:2229] T 55a50b1b61934f4d8b4b6d79e908a299 P f03faf52a51442b997fbefd7943ee930 [term 1 LEADER]: Raft consensus shutting down.
I20260808 08:25:19.488009 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 31180
I20260808 08:25:19.488051 31935 raft_consensus.cc:2258] T 55a50b1b61934f4d8b4b6d79e908a299 P f03faf52a51442b997fbefd7943ee930 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:25:19.488371 31936 tablet_replica.cc:330] T 55a50b1b61934f4d8b4b6d79e908a299 P ac00006170914cafb093d44089e5e06d: stopping tablet replica
I20260808 08:25:19.488677 31936 raft_consensus.cc:2229] T 55a50b1b61934f4d8b4b6d79e908a299 P ac00006170914cafb093d44089e5e06d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 08:25:19.489025 31936 raft_consensus.cc:2258] T 55a50b1b61934f4d8b4b6d79e908a299 P ac00006170914cafb093d44089e5e06d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:25:19.492415 31935 ts_tablet_manager.cc:1905] T 55a50b1b61934f4d8b4b6d79e908a299 P f03faf52a51442b997fbefd7943ee930: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:25:19.493372 31936 ts_tablet_manager.cc:1905] T 55a50b1b61934f4d8b4b6d79e908a299 P ac00006170914cafb093d44089e5e06d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:25:19.495338 31935 ts_tablet_manager.cc:1918] T 55a50b1b61934f4d8b4b6d79e908a299 P f03faf52a51442b997fbefd7943ee930: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 08:25:19.495458 31935 log.cc:1196] T 55a50b1b61934f4d8b4b6d79e908a299 P f03faf52a51442b997fbefd7943ee930: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-2/wal/wals/55a50b1b61934f4d8b4b6d79e908a299
I20260808 08:25:19.495850 31935 ts_tablet_manager.cc:1939] T 55a50b1b61934f4d8b4b6d79e908a299 P f03faf52a51442b997fbefd7943ee930: Deleting consensus metadata
I20260808 08:25:19.497382 31064 catalog_manager.cc:4841] TS f03faf52a51442b997fbefd7943ee930 (127.22.235.195:37793): tablet 55a50b1b61934f4d8b4b6d79e908a299 (table test-table [id=023488df763140338aaedd7f0a0722cd]) successfully deleted
I20260808 08:25:19.497423 31936 ts_tablet_manager.cc:1918] T 55a50b1b61934f4d8b4b6d79e908a299 P ac00006170914cafb093d44089e5e06d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 08:25:19.497641 31936 log.cc:1196] T 55a50b1b61934f4d8b4b6d79e908a299 P ac00006170914cafb093d44089e5e06d: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786177493204855-23471-0/minicluster-data/ts-1/wal/wals/55a50b1b61934f4d8b4b6d79e908a299
I20260808 08:25:19.497944 31936 ts_tablet_manager.cc:1939] T 55a50b1b61934f4d8b4b6d79e908a299 P ac00006170914cafb093d44089e5e06d: Deleting consensus metadata
I20260808 08:25:19.499083 31066 catalog_manager.cc:4841] TS ac00006170914cafb093d44089e5e06d (127.22.235.194:38781): tablet 55a50b1b61934f4d8b4b6d79e908a299 (table test-table [id=023488df763140338aaedd7f0a0722cd]) successfully deleted
W20260808 08:25:19.503595 31067 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.235.193:34643 (error 108)
I20260808 08:25:19.504102 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 31419
I20260808 08:25:19.515441 31891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:19.515104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:34643 (local address 127.0.0.1:53104)
0808 08:25:19.515255 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:19.515332 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.193:34643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260808 08:25:19.516759 31067 catalog_manager.cc:4584] TS 101d342b95324662ab7b4bebfdab00c2 (127.22.235.193:34643): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 55a50b1b61934f4d8b4b6d79e908a299: Network error: Client connection negotiation failed: client connection to 127.22.235.193:34643: connect: Connection refused (error 111)
I20260808 08:25:19.522317 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 31659
I20260808 08:25:19.539750 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 31026
2026-08-08T08:25:19Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1181 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260808 08:25:19.566339 23471 mini_oidc.cc:152] Starting JWKS server
I20260808 08:25:19.568123 23471 webserver.cc:460] Webserver started at https://127.0.0.1:33483/ using document root <none> and password file <none>
I20260808 08:25:19.568316 23471 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 08:25:19.568673 23471 webserver.cc:460] Webserver started at http://127.0.0.1:37509/ using document root <none> and password file <none>
2026-08-08T08:25:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:25:19Z Disabled control of system clock
I20260808 08:25:19.577957 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:44177
--webserver_interface=127.22.235.254
--webserver_port=0
--builtin_ntp_servers=127.22.235.212:45281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:44177
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33483/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786177493204855-23471-0/testchainca.cert with env {}
W20260808 08:25:19.686955 31946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:19.687191 31946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:19.687229 31946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:19.691284 31946 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:25:19.691339 31946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:19.691354 31946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:19.691372 31946 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:25:19.691401 31946 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 08:25:19.691425 31946 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 08:25:19.691449 31946 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33483/jwks.json
I20260808 08:25:19.696578 31946 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:45281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:44177
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:44177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33483/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.235.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786177493204855-23471-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31946
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:19.697590 31946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:19.698534 31946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:19.706426 31959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:19.706439 31962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:19.706643 31946 server_base.cc:1034] running on GCE node
W20260808 08:25:19.706457 31958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:19.707553 31946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:19.708572 31946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:19.709767 31946 hybrid_clock.cc:648] HybridClock initialized: now 1786177519709751 us; error 27 us; skew 500 ppm
I20260808 08:25:19.711462 31946 webserver.cc:460] Webserver started at http://127.22.235.254:32985/ using document root <none> and password file <none>
I20260808 08:25:19.712004 31946 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:19.712071 31946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:19.712256 31946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:19.714211 31946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "843b5816634b4a4b92c44be0abfbd09d"
format_stamp: "Formatted at 2026-08-08 08:25:19 on dist-test-slave-f7th"
server_key: "c83a44439d33f692c04fc63f43e2a673"
server_key_iv: "bf42985870abe13e0d3d1cdf88a7a872"
server_key_version: "encryptionkey@0"
I20260808 08:25:19.714715 31946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "843b5816634b4a4b92c44be0abfbd09d"
format_stamp: "Formatted at 2026-08-08 08:25:19 on dist-test-slave-f7th"
server_key: "c83a44439d33f692c04fc63f43e2a673"
server_key_iv: "bf42985870abe13e0d3d1cdf88a7a872"
server_key_version: "encryptionkey@0"
I20260808 08:25:19.718182 31946 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 08:25:19.720471 31970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:19.721618 31946 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:25:19.721742 31946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "843b5816634b4a4b92c44be0abfbd09d"
format_stamp: "Formatted at 2026-08-08 08:25:19 on dist-test-slave-f7th"
server_key: "c83a44439d33f692c04fc63f43e2a673"
server_key_iv: "bf42985870abe13e0d3d1cdf88a7a872"
server_key_version: "encryptionkey@0"
I20260808 08:25:19.721830 31946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:19.742431 31946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:19.743076 31946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:19.743243 31946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:19.749908 31946 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.254:44177
I20260808 08:25:19.749922 32060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.254:44177 every 8 connection(s)
I20260808 08:25:19.750828 31946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:25:19.752833 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 31946
I20260808 08:25:19.752961 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:25:19.753260 23471 external_mini_cluster.cc:1428] Setting key e2106e69b719dcb8ea65ec1569c88c59
I20260808 08:25:19.754437 32062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:19.760102 32062 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 843b5816634b4a4b92c44be0abfbd09d: Bootstrap starting.
I20260808 08:25:19.762001 32070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:19.754599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55896 (local address 127.22.235.254:44177)
0808 08:25:19.755235 (+ 636us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:19.755250 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:19.755284 (+ 34us) server_negotiation.cc:407] Connection header received
0808 08:25:19.755913 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:19.755928 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:19.756319 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:19.756648 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:19.757072 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:19.758011 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:19.758899 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:19.759354 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:19.759502 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:19.760343 (+ 841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:19.760348 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:19.760361 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:25:19.760772 (+ 411us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:19.760836 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:25:19.760857 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:19.760948 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:19.761081 (+ 133us) server_negotiation.cc:299] Negotiation successful
0808 08:25:19.761323 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":138,"threads_started":1}
I20260808 08:25:19.762434 32062 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 843b5816634b4a4b92c44be0abfbd09d: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:19.763231 32062 log.cc:824] T 00000000000000000000000000000000 P 843b5816634b4a4b92c44be0abfbd09d: Log is configured to *not* fsync() on all Append() calls
I20260808 08:25:19.764781 23471 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 08:25:19.765311 32062 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 843b5816634b4a4b92c44be0abfbd09d: No bootstrap required, opened a new log
I20260808 08:25:19.768718 32062 raft_consensus.cc:348] T 00000000000000000000000000000000 P 843b5816634b4a4b92c44be0abfbd09d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843b5816634b4a4b92c44be0abfbd09d" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 44177 } }
I20260808 08:25:19.768925 32062 raft_consensus.cc:374] T 00000000000000000000000000000000 P 843b5816634b4a4b92c44be0abfbd09d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:19.768971 32062 raft_consensus.cc:729] T 00000000000000000000000000000000 P 843b5816634b4a4b92c44be0abfbd09d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843b5816634b4a4b92c44be0abfbd09d, State: Initialized, Role: FOLLOWER
I20260808 08:25:19.769317 32062 consensus_queue.cc:260] T 00000000000000000000000000000000 P 843b5816634b4a4b92c44be0abfbd09d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843b5816634b4a4b92c44be0abfbd09d" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 44177 } }
I20260808 08:25:19.769423 32062 raft_consensus.cc:388] T 00000000000000000000000000000000 P 843b5816634b4a4b92c44be0abfbd09d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:25:19.769475 32062 raft_consensus.cc:482] T 00000000000000000000000000000000 P 843b5816634b4a4b92c44be0abfbd09d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:25:19.769544 32062 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 843b5816634b4a4b92c44be0abfbd09d [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:19.770303 32062 raft_consensus.cc:504] T 00000000000000000000000000000000 P 843b5816634b4a4b92c44be0abfbd09d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843b5816634b4a4b92c44be0abfbd09d" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 44177 } }
I20260808 08:25:19.770596 32062 leader_election.cc:302] T 00000000000000000000000000000000 P 843b5816634b4a4b92c44be0abfbd09d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 843b5816634b4a4b92c44be0abfbd09d; no voters:
I20260808 08:25:19.770892 32062 leader_election.cc:288] T 00000000000000000000000000000000 P 843b5816634b4a4b92c44be0abfbd09d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:25:19.770974 32072 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 843b5816634b4a4b92c44be0abfbd09d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:25:19.771209 32072 raft_consensus.cc:686] T 00000000000000000000000000000000 P 843b5816634b4a4b92c44be0abfbd09d [term 1 LEADER]: Becoming Leader. State: Replica: 843b5816634b4a4b92c44be0abfbd09d, State: Running, Role: LEADER
I20260808 08:25:19.771512 32072 consensus_queue.cc:237] T 00000000000000000000000000000000 P 843b5816634b4a4b92c44be0abfbd09d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843b5816634b4a4b92c44be0abfbd09d" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 44177 } }
I20260808 08:25:19.772100 32062 sys_catalog.cc:564] T 00000000000000000000000000000000 P 843b5816634b4a4b92c44be0abfbd09d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:25:19.773124 32077 sys_catalog.cc:455] T 00000000000000000000000000000000 P 843b5816634b4a4b92c44be0abfbd09d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 843b5816634b4a4b92c44be0abfbd09d. Latest consensus state: current_term: 1 leader_uuid: "843b5816634b4a4b92c44be0abfbd09d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843b5816634b4a4b92c44be0abfbd09d" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 44177 } } }
I20260808 08:25:19.773100 32075 sys_catalog.cc:455] T 00000000000000000000000000000000 P 843b5816634b4a4b92c44be0abfbd09d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "843b5816634b4a4b92c44be0abfbd09d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843b5816634b4a4b92c44be0abfbd09d" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 44177 } } }
I20260808 08:25:19.773334 32075 sys_catalog.cc:458] T 00000000000000000000000000000000 P 843b5816634b4a4b92c44be0abfbd09d [sys.catalog]: This master's current role is: LEADER
I20260808 08:25:19.773257 32077 sys_catalog.cc:458] T 00000000000000000000000000000000 P 843b5816634b4a4b92c44be0abfbd09d [sys.catalog]: This master's current role is: LEADER
I20260808 08:25:19.773766 32089 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:25:19.776897 32089 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:25:19.782374 32089 catalog_manager.cc:1347] Generated new cluster ID: 8b9433fb715e40008326e6473156dfc5
I20260808 08:25:19.782435 32089 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:25:19.790503 32089 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:25:19.791316 32089 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:25:19.795301 32089 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 843b5816634b4a4b92c44be0abfbd09d: Generated new TSK 0
I20260808 08:25:19.795928 32089 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 08:25:19.822731 32114 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260808 08:25:19.822870 32114 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 08:25:19.823485 32070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:19.806218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55898 (local address 127.22.235.254:44177)
0808 08:25:19.806348 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:19.806353 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:19.806516 (+ 163us) server_negotiation.cc:407] Connection header received
0808 08:25:19.806701 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:19.806704 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:19.806758 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:19.806871 (+ 113us) server_negotiation.cc:226] Negotiated authn=JWT
0808 08:25:19.807096 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:19.807902 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:19.808761 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:19.809110 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:19.809210 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:19.809480 (+ 270us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 08:25:19.823081 (+ 13601us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33483/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 08:25:19.823349 (+ 268us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55898: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33483/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":44}
W20260808 08:25:19.823566 32070 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55898: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33483/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260808 08:25:19.824606 32113 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:19.806119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:44177 (local address 127.0.0.1:55898)
0808 08:25:19.806446 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:19.806457 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:19.806561 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:19.806884 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:19.806886 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:19.806892 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0808 08:25:19.807007 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:19.807012 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:19.808014 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:19.808016 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:19.808663 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:19.808673 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:19.809207 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:19.809209 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:19.809275 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:19.809318 (+ 43us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 08:25:19.823273 (+ 13955us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33483/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 08:25:19.823453 (+ 180us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.22.235.254:44177: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33483/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":107,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.22.235.254:44177: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33483/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260808 08:25:19.825552 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 31946
2026-08-08T08:25:19Z chronyd exiting
I20260808 08:25:19.870563 23471 test_util.cc:175] -----------------------------------------------
I20260808 08:25:19.870651 23471 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786177493204855-23471-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (305 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260808 08:25:19.871762 23471 mini_oidc.cc:152] Starting JWKS server
I20260808 08:25:19.872571 23471 webserver.cc:460] Webserver started at https://127.0.0.1:37229/ using document root <none> and password file <none>
I20260808 08:25:19.872689 23471 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 08:25:19.872925 23471 webserver.cc:460] Webserver started at http://127.0.0.1:39875/ using document root <none> and password file <none>
2026-08-08T08:25:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:25:19Z Disabled control of system clock
I20260808 08:25:19.882149 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:42149
--webserver_interface=127.22.235.254
--webserver_port=0
--builtin_ntp_servers=127.22.235.212:34055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:42149
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37229/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786177493204855-23471-0/testchainca.cert with env {}
W20260808 08:25:19.991040 32121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:19.991288 32121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:19.991339 32121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:19.995223 32121 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:25:19.995276 32121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:19.995297 32121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:19.995318 32121 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:25:19.995347 32121 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 08:25:19.995375 32121 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 08:25:19.995395 32121 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37229/jwks.json
I20260808 08:25:20.000550 32121 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:34055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:42149
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:42149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37229/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.235.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786177493204855-23471-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32121
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:20.001569 32121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:20.002599 32121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 08:25:20.010893 32121 server_base.cc:1034] running on GCE node
W20260808 08:25:20.010829 32130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:20.010838 32127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:20.010872 32133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:20.011708 32121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:20.012871 32121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:20.014032 32121 hybrid_clock.cc:648] HybridClock initialized: now 1786177520014016 us; error 27 us; skew 500 ppm
I20260808 08:25:20.015846 32121 webserver.cc:460] Webserver started at http://127.22.235.254:37793/ using document root <none> and password file <none>
I20260808 08:25:20.016480 32121 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:20.016552 32121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:20.016790 32121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:20.018779 32121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "ab4a5fc81a5b414dabb2db4033aacab3"
format_stamp: "Formatted at 2026-08-08 08:25:20 on dist-test-slave-f7th"
server_key: "b2e0ea24a13bf7447b9d6f1b49d87595"
server_key_iv: "fa7052c6eca6a05c684beb4a881c3994"
server_key_version: "encryptionkey@0"
I20260808 08:25:20.019317 32121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "ab4a5fc81a5b414dabb2db4033aacab3"
format_stamp: "Formatted at 2026-08-08 08:25:20 on dist-test-slave-f7th"
server_key: "b2e0ea24a13bf7447b9d6f1b49d87595"
server_key_iv: "fa7052c6eca6a05c684beb4a881c3994"
server_key_version: "encryptionkey@0"
I20260808 08:25:20.022785 32121 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 08:25:20.025122 32141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:20.026278 32121 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:25:20.026393 32121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "ab4a5fc81a5b414dabb2db4033aacab3"
format_stamp: "Formatted at 2026-08-08 08:25:20 on dist-test-slave-f7th"
server_key: "b2e0ea24a13bf7447b9d6f1b49d87595"
server_key_iv: "fa7052c6eca6a05c684beb4a881c3994"
server_key_version: "encryptionkey@0"
I20260808 08:25:20.026487 32121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:20.045431 32121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:20.046089 32121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:20.046262 32121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:20.052745 32121 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.254:42149
I20260808 08:25:20.052758 32240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.254:42149 every 8 connection(s)
I20260808 08:25:20.053656 32121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:25:20.056715 32241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:20.057125 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 32121
I20260808 08:25:20.057250 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:25:20.057565 23471 external_mini_cluster.cc:1428] Setting key 98cac00e8b11dd6e51b7453163f25fbf
I20260808 08:25:20.062692 32241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab4a5fc81a5b414dabb2db4033aacab3: Bootstrap starting.
I20260808 08:25:20.064946 32241 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab4a5fc81a5b414dabb2db4033aacab3: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:20.065701 32241 log.cc:824] T 00000000000000000000000000000000 P ab4a5fc81a5b414dabb2db4033aacab3: Log is configured to *not* fsync() on all Append() calls
I20260808 08:25:20.066519 32245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:20.058912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45030 (local address 127.22.235.254:42149)
0808 08:25:20.059529 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:20.059543 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:20.059577 (+ 34us) server_negotiation.cc:407] Connection header received
0808 08:25:20.060229 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:20.060246 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:20.060676 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:20.061002 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:20.061412 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:20.062356 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:20.063264 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:20.063722 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:20.063869 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:20.064719 (+ 850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:20.064724 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:20.064738 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:25:20.065135 (+ 397us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:20.065197 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:25:20.065218 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:20.065306 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:20.065441 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 08:25:20.065692 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":120,"threads_started":1}
I20260808 08:25:20.067906 32241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab4a5fc81a5b414dabb2db4033aacab3: No bootstrap required, opened a new log
I20260808 08:25:20.070678 23471 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 08:25:20.072345 32241 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab4a5fc81a5b414dabb2db4033aacab3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab4a5fc81a5b414dabb2db4033aacab3" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 42149 } }
I20260808 08:25:20.072597 32241 raft_consensus.cc:374] T 00000000000000000000000000000000 P ab4a5fc81a5b414dabb2db4033aacab3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:20.072675 32241 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab4a5fc81a5b414dabb2db4033aacab3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab4a5fc81a5b414dabb2db4033aacab3, State: Initialized, Role: FOLLOWER
I20260808 08:25:20.073071 32241 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab4a5fc81a5b414dabb2db4033aacab3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab4a5fc81a5b414dabb2db4033aacab3" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 42149 } }
I20260808 08:25:20.073205 32241 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab4a5fc81a5b414dabb2db4033aacab3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:25:20.073262 32241 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab4a5fc81a5b414dabb2db4033aacab3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:25:20.073335 32241 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab4a5fc81a5b414dabb2db4033aacab3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:20.074470 32241 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab4a5fc81a5b414dabb2db4033aacab3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab4a5fc81a5b414dabb2db4033aacab3" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 42149 } }
I20260808 08:25:20.074779 32241 leader_election.cc:302] T 00000000000000000000000000000000 P ab4a5fc81a5b414dabb2db4033aacab3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab4a5fc81a5b414dabb2db4033aacab3; no voters:
I20260808 08:25:20.075109 32241 leader_election.cc:288] T 00000000000000000000000000000000 P ab4a5fc81a5b414dabb2db4033aacab3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:25:20.075224 32247 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab4a5fc81a5b414dabb2db4033aacab3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:25:20.075420 32247 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab4a5fc81a5b414dabb2db4033aacab3 [term 1 LEADER]: Becoming Leader. State: Replica: ab4a5fc81a5b414dabb2db4033aacab3, State: Running, Role: LEADER
I20260808 08:25:20.075731 32247 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab4a5fc81a5b414dabb2db4033aacab3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab4a5fc81a5b414dabb2db4033aacab3" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 42149 } }
I20260808 08:25:20.076306 32241 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab4a5fc81a5b414dabb2db4033aacab3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:25:20.077330 32248 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab4a5fc81a5b414dabb2db4033aacab3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ab4a5fc81a5b414dabb2db4033aacab3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab4a5fc81a5b414dabb2db4033aacab3" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 42149 } } }
I20260808 08:25:20.077468 32248 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab4a5fc81a5b414dabb2db4033aacab3 [sys.catalog]: This master's current role is: LEADER
I20260808 08:25:20.077730 32249 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab4a5fc81a5b414dabb2db4033aacab3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab4a5fc81a5b414dabb2db4033aacab3. Latest consensus state: current_term: 1 leader_uuid: "ab4a5fc81a5b414dabb2db4033aacab3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab4a5fc81a5b414dabb2db4033aacab3" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 42149 } } }
I20260808 08:25:20.077834 32249 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab4a5fc81a5b414dabb2db4033aacab3 [sys.catalog]: This master's current role is: LEADER
I20260808 08:25:20.077942 32259 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:25:20.081238 32259 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:25:20.086803 32259 catalog_manager.cc:1347] Generated new cluster ID: 929b6944ab1748498fe9194b51a6f564
I20260808 08:25:20.086877 32259 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:25:20.102840 32259 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:25:20.104058 32259 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:25:20.111804 32259 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ab4a5fc81a5b414dabb2db4033aacab3: Generated new TSK 0
I20260808 08:25:20.112584 32259 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 08:25:20.138411 32291 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260808 08:25:20.138486 32291 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 08:25:20.139168 32245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:20.122592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45044 (local address 127.22.235.254:42149)
0808 08:25:20.122726 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:20.122731 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:20.122847 (+ 116us) server_negotiation.cc:407] Connection header received
0808 08:25:20.122981 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:20.122984 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:20.123039 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:20.123155 (+ 116us) server_negotiation.cc:226] Negotiated authn=JWT
0808 08:25:20.123325 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:20.124164 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:20.125149 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:20.125510 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:20.125611 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:20.125898 (+ 287us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 08:25:20.138757 (+ 12859us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37229/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 08:25:20.139031 (+ 274us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45044: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37229/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":64}
W20260808 08:25:20.139237 32245 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45044: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37229/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260808 08:25:20.139277 32289 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:20.122491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42149 (local address 127.0.0.1:45044)
0808 08:25:20.122763 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:20.122775 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:20.122867 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:20.123144 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:20.123146 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:20.123152 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0808 08:25:20.123242 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:20.123246 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:20.124349 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:20.124352 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:20.125026 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:20.125036 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:20.125634 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:20.125637 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:20.125706 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:20.125729 (+ 23us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 08:25:20.138973 (+ 13244us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37229/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 08:25:20.139126 (+ 153us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.22.235.254:42149: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37229/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":89,"threads_started":1}
I20260808 08:25:20.139930 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 32121
2026-08-08T08:25:20Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (317 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260808 08:25:20.189332 23471 mini_oidc.cc:152] Starting JWKS server
I20260808 08:25:20.190188 23471 webserver.cc:460] Webserver started at https://127.0.0.1:45639/ using document root <none> and password file <none>
I20260808 08:25:20.190285 23471 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 08:25:20.190541 23471 webserver.cc:460] Webserver started at http://127.0.0.1:46763/ using document root <none> and password file <none>
2026-08-08T08:25:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:25:20Z Disabled control of system clock
I20260808 08:25:20.200160 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:37809
--webserver_interface=127.22.235.254
--webserver_port=0
--builtin_ntp_servers=127.22.235.212:46271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:37809
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45639/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 08:25:20.309903 32301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:20.310149 32301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:20.310185 32301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:20.314085 32301 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:25:20.314136 32301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:20.314152 32301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:20.314173 32301 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:25:20.314190 32301 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 08:25:20.314217 32301 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 08:25:20.314244 32301 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45639/jwks.json
I20260808 08:25:20.318874 32301 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:46271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:37809
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:37809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45639/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.235.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:20.319810 32301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:20.321156 32301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:20.329079 32309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:20.329094 32307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:20.329079 32313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:20.329339 32301 server_base.cc:1034] running on GCE node
I20260808 08:25:20.329896 32301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:20.330894 32301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:20.332050 32301 hybrid_clock.cc:648] HybridClock initialized: now 1786177520332036 us; error 26 us; skew 500 ppm
I20260808 08:25:20.333755 32301 webserver.cc:460] Webserver started at http://127.22.235.254:45887/ using document root <none> and password file <none>
I20260808 08:25:20.334290 32301 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:20.334350 32301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:20.334550 32301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:20.336513 32301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "07e44b47377a45cfbd166ed6daf32f0c"
format_stamp: "Formatted at 2026-08-08 08:25:20 on dist-test-slave-f7th"
server_key: "3ff96af107564bb4c640f150eeea1e9b"
server_key_iv: "60399a4f9f5fab31c75f984520be3328"
server_key_version: "encryptionkey@0"
I20260808 08:25:20.337067 32301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "07e44b47377a45cfbd166ed6daf32f0c"
format_stamp: "Formatted at 2026-08-08 08:25:20 on dist-test-slave-f7th"
server_key: "3ff96af107564bb4c640f150eeea1e9b"
server_key_iv: "60399a4f9f5fab31c75f984520be3328"
server_key_version: "encryptionkey@0"
I20260808 08:25:20.340508 32301 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 08:25:20.342690 32322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:20.343792 32301 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:25:20.343914 32301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "07e44b47377a45cfbd166ed6daf32f0c"
format_stamp: "Formatted at 2026-08-08 08:25:20 on dist-test-slave-f7th"
server_key: "3ff96af107564bb4c640f150eeea1e9b"
server_key_iv: "60399a4f9f5fab31c75f984520be3328"
server_key_version: "encryptionkey@0"
I20260808 08:25:20.344002 32301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:20.356986 32301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:20.357648 32301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:20.357812 32301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:20.364238 32301 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.254:37809
I20260808 08:25:20.364249 32413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.254:37809 every 8 connection(s)
I20260808 08:25:20.365170 32301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:25:20.367916 32414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:20.373298 32414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07e44b47377a45cfbd166ed6daf32f0c: Bootstrap starting.
I20260808 08:25:20.375161 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 32301
I20260808 08:25:20.375273 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:25:20.375510 23471 external_mini_cluster.cc:1428] Setting key 15d340db2d7c619eec6adb7ac4c034b1
I20260808 08:25:20.375721 32414 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 07e44b47377a45cfbd166ed6daf32f0c: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:20.376585 32414 log.cc:824] T 00000000000000000000000000000000 P 07e44b47377a45cfbd166ed6daf32f0c: Log is configured to *not* fsync() on all Append() calls
I20260808 08:25:20.378494 32414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07e44b47377a45cfbd166ed6daf32f0c: No bootstrap required, opened a new log
I20260808 08:25:20.381901 32414 raft_consensus.cc:348] T 00000000000000000000000000000000 P 07e44b47377a45cfbd166ed6daf32f0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07e44b47377a45cfbd166ed6daf32f0c" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 37809 } }
I20260808 08:25:20.382169 32414 raft_consensus.cc:374] T 00000000000000000000000000000000 P 07e44b47377a45cfbd166ed6daf32f0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:20.382244 32414 raft_consensus.cc:729] T 00000000000000000000000000000000 P 07e44b47377a45cfbd166ed6daf32f0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07e44b47377a45cfbd166ed6daf32f0c, State: Initialized, Role: FOLLOWER
I20260808 08:25:20.382686 32414 consensus_queue.cc:260] T 00000000000000000000000000000000 P 07e44b47377a45cfbd166ed6daf32f0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07e44b47377a45cfbd166ed6daf32f0c" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 37809 } }
I20260808 08:25:20.382819 32414 raft_consensus.cc:388] T 00000000000000000000000000000000 P 07e44b47377a45cfbd166ed6daf32f0c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:25:20.382886 32414 raft_consensus.cc:482] T 00000000000000000000000000000000 P 07e44b47377a45cfbd166ed6daf32f0c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:25:20.382980 32414 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 07e44b47377a45cfbd166ed6daf32f0c [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:20.383869 32414 raft_consensus.cc:504] T 00000000000000000000000000000000 P 07e44b47377a45cfbd166ed6daf32f0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07e44b47377a45cfbd166ed6daf32f0c" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 37809 } }
I20260808 08:25:20.383982 32420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:20.376530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60542 (local address 127.22.235.254:37809)
0808 08:25:20.377245 (+ 715us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:20.377266 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:20.377309 (+ 43us) server_negotiation.cc:407] Connection header received
0808 08:25:20.377949 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:20.377965 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:20.378289 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:20.378584 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:20.379015 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:20.379951 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:20.380888 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:20.381353 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:20.381502 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:20.382332 (+ 830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:20.382339 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:20.382357 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:25:20.382727 (+ 370us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:20.382781 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:25:20.382801 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:20.382887 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:20.383065 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 08:25:20.383308 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":131,"threads_started":1}
I20260808 08:25:20.384225 32414 leader_election.cc:302] T 00000000000000000000000000000000 P 07e44b47377a45cfbd166ed6daf32f0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 07e44b47377a45cfbd166ed6daf32f0c; no voters:
I20260808 08:25:20.384559 32414 leader_election.cc:288] T 00000000000000000000000000000000 P 07e44b47377a45cfbd166ed6daf32f0c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:25:20.384672 32425 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 07e44b47377a45cfbd166ed6daf32f0c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:25:20.384864 32425 raft_consensus.cc:686] T 00000000000000000000000000000000 P 07e44b47377a45cfbd166ed6daf32f0c [term 1 LEADER]: Becoming Leader. State: Replica: 07e44b47377a45cfbd166ed6daf32f0c, State: Running, Role: LEADER
I20260808 08:25:20.385157 32425 consensus_queue.cc:237] T 00000000000000000000000000000000 P 07e44b47377a45cfbd166ed6daf32f0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07e44b47377a45cfbd166ed6daf32f0c" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 37809 } }
I20260808 08:25:20.385805 32414 sys_catalog.cc:564] T 00000000000000000000000000000000 P 07e44b47377a45cfbd166ed6daf32f0c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:25:20.386710 32429 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07e44b47377a45cfbd166ed6daf32f0c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "07e44b47377a45cfbd166ed6daf32f0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07e44b47377a45cfbd166ed6daf32f0c" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 37809 } } }
I20260808 08:25:20.386845 32429 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07e44b47377a45cfbd166ed6daf32f0c [sys.catalog]: This master's current role is: LEADER
I20260808 08:25:20.387101 23471 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 08:25:20.388043 32441 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:25:20.388767 32431 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07e44b47377a45cfbd166ed6daf32f0c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 07e44b47377a45cfbd166ed6daf32f0c. Latest consensus state: current_term: 1 leader_uuid: "07e44b47377a45cfbd166ed6daf32f0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07e44b47377a45cfbd166ed6daf32f0c" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 37809 } } }
I20260808 08:25:20.388895 32431 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07e44b47377a45cfbd166ed6daf32f0c [sys.catalog]: This master's current role is: LEADER
I20260808 08:25:20.391180 32441 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:25:20.396098 32441 catalog_manager.cc:1347] Generated new cluster ID: dd284c66857c453d875fc27363742ab1
I20260808 08:25:20.396162 32441 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:25:20.413158 32441 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:25:20.414026 32441 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:25:20.421916 32441 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 07e44b47377a45cfbd166ed6daf32f0c: Generated new TSK 0
I20260808 08:25:20.422554 32441 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 08:25:20.437831 32420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:20.432998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60556 (local address 127.22.235.254:37809)
0808 08:25:20.433115 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:20.433120 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:20.433219 (+ 99us) server_negotiation.cc:407] Connection header received
0808 08:25:20.433363 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:20.433366 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:20.433422 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:20.433535 (+ 113us) server_negotiation.cc:226] Negotiated authn=JWT
0808 08:25:20.433720 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:20.434540 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:20.435356 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:20.435697 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:20.435790 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:20.436038 (+ 248us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 08:25:20.437465 (+ 1427us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45639/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 08:25:20.437690 (+ 225us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60556: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45639/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":46}
W20260808 08:25:20.437901 32420 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60556: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45639/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260808 08:25:20.439354 32463 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:20.432874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:37809 (local address 127.0.0.1:60556)
0808 08:25:20.433138 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:20.433150 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:20.433245 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:20.433534 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:20.433536 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:20.433542 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0808 08:25:20.433631 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:20.433636 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:20.434630 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:20.434633 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:20.435255 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:20.435262 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:20.435797 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:20.435800 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:20.435858 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:20.435879 (+ 21us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 08:25:20.437621 (+ 1742us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45639/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 08:25:20.439191 (+ 1570us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.22.235.254:37809: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45639/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":1396,"thread_start_us":89,"threads_started":1}
I20260808 08:25:20.439872 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 32301
E20260808 08:25:20.453277 32467 webserver.cc:533] Webserver: sslize failed:
E20260808 08:25:20.453372 32467 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-08T08:25:20Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (298 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-08T08:25:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:25:20Z Disabled control of system clock
I20260808 08:25:20.496567 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:46089
--webserver_interface=127.22.235.254
--webserver_port=0
--builtin_ntp_servers=127.22.235.212:37317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:46089
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786177493204855-23471-0/testchainca.cert with env {}
W20260808 08:25:20.605628 32472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:20.605883 32472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:20.605919 32472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:20.610190 32472 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:25:20.610261 32472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:20.610276 32472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:20.610293 32472 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:25:20.610318 32472 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 08:25:20.610349 32472 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 08:25:20.610376 32472 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260808 08:25:20.615862 32472 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:37317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:46089
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:46089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.235.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786177493204855-23471-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:20.616925 32472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:20.617949 32472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:20.626327 32482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:20.626385 32480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:20.626348 32479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:20.626576 32472 server_base.cc:1034] running on GCE node
I20260808 08:25:20.627210 32472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:20.628319 32472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:20.629500 32472 hybrid_clock.cc:648] HybridClock initialized: now 1786177520629485 us; error 29 us; skew 500 ppm
I20260808 08:25:20.631317 32472 webserver.cc:460] Webserver started at http://127.22.235.254:34293/ using document root <none> and password file <none>
I20260808 08:25:20.631891 32472 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:20.631960 32472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:20.632153 32472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:20.634186 32472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "ce7c7c06be0846efa867e88f0c5ff7b1"
format_stamp: "Formatted at 2026-08-08 08:25:20 on dist-test-slave-f7th"
server_key: "ed56450d1171f4fe6f92e65094d1e15c"
server_key_iv: "9e9b1a77dba96b53e29b846b5b1c35f8"
server_key_version: "encryptionkey@0"
I20260808 08:25:20.634706 32472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "ce7c7c06be0846efa867e88f0c5ff7b1"
format_stamp: "Formatted at 2026-08-08 08:25:20 on dist-test-slave-f7th"
server_key: "ed56450d1171f4fe6f92e65094d1e15c"
server_key_iv: "9e9b1a77dba96b53e29b846b5b1c35f8"
server_key_version: "encryptionkey@0"
I20260808 08:25:20.638190 32472 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 08:25:20.640439 32491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:20.641577 32472 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:25:20.641690 32472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "ce7c7c06be0846efa867e88f0c5ff7b1"
format_stamp: "Formatted at 2026-08-08 08:25:20 on dist-test-slave-f7th"
server_key: "ed56450d1171f4fe6f92e65094d1e15c"
server_key_iv: "9e9b1a77dba96b53e29b846b5b1c35f8"
server_key_version: "encryptionkey@0"
I20260808 08:25:20.641779 32472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:20.675643 32472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:20.676350 32472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:20.676529 32472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:20.683109 32472 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.254:46089
I20260808 08:25:20.683125 32584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.254:46089 every 8 connection(s)
I20260808 08:25:20.684046 32472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:25:20.686934 32586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:20.691931 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 32472
I20260808 08:25:20.692051 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:25:20.692345 23471 external_mini_cluster.cc:1428] Setting key c77c6f273b5bded445b8cc7abefbcb76
I20260808 08:25:20.692844 32586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce7c7c06be0846efa867e88f0c5ff7b1: Bootstrap starting.
I20260808 08:25:20.695256 32586 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce7c7c06be0846efa867e88f0c5ff7b1: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:20.696085 32586 log.cc:824] T 00000000000000000000000000000000 P ce7c7c06be0846efa867e88f0c5ff7b1: Log is configured to *not* fsync() on all Append() calls
I20260808 08:25:20.698179 32586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce7c7c06be0846efa867e88f0c5ff7b1: No bootstrap required, opened a new log
I20260808 08:25:20.701375 32592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:20.693764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47576 (local address 127.22.235.254:46089)
0808 08:25:20.694359 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:20.694374 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:20.694410 (+ 36us) server_negotiation.cc:407] Connection header received
0808 08:25:20.695014 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:20.695030 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:20.695454 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:20.695727 (+ 273us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:20.696072 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:20.697209 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:20.698347 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:20.698788 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:20.698910 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:20.699708 (+ 798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:20.699713 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:20.699726 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:25:20.700103 (+ 377us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:20.700164 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:25:20.700184 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:20.700255 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:20.700400 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 08:25:20.700675 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":96,"threads_started":1}
I20260808 08:25:20.701989 32586 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce7c7c06be0846efa867e88f0c5ff7b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce7c7c06be0846efa867e88f0c5ff7b1" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46089 } }
I20260808 08:25:20.702194 32586 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce7c7c06be0846efa867e88f0c5ff7b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:20.702232 32586 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce7c7c06be0846efa867e88f0c5ff7b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce7c7c06be0846efa867e88f0c5ff7b1, State: Initialized, Role: FOLLOWER
I20260808 08:25:20.702685 32586 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce7c7c06be0846efa867e88f0c5ff7b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce7c7c06be0846efa867e88f0c5ff7b1" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46089 } }
I20260808 08:25:20.702841 32586 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce7c7c06be0846efa867e88f0c5ff7b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:25:20.702905 32586 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce7c7c06be0846efa867e88f0c5ff7b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:25:20.703006 32586 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce7c7c06be0846efa867e88f0c5ff7b1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:20.703933 32586 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce7c7c06be0846efa867e88f0c5ff7b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce7c7c06be0846efa867e88f0c5ff7b1" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46089 } }
I20260808 08:25:20.704303 23471 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 08:25:20.704276 32586 leader_election.cc:302] T 00000000000000000000000000000000 P ce7c7c06be0846efa867e88f0c5ff7b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce7c7c06be0846efa867e88f0c5ff7b1; no voters:
I20260808 08:25:20.704588 32586 leader_election.cc:288] T 00000000000000000000000000000000 P ce7c7c06be0846efa867e88f0c5ff7b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:25:20.704687 32595 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce7c7c06be0846efa867e88f0c5ff7b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:25:20.704877 32595 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce7c7c06be0846efa867e88f0c5ff7b1 [term 1 LEADER]: Becoming Leader. State: Replica: ce7c7c06be0846efa867e88f0c5ff7b1, State: Running, Role: LEADER
I20260808 08:25:20.705173 32595 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce7c7c06be0846efa867e88f0c5ff7b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce7c7c06be0846efa867e88f0c5ff7b1" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46089 } }
I20260808 08:25:20.705844 32586 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce7c7c06be0846efa867e88f0c5ff7b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:25:20.706844 32596 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce7c7c06be0846efa867e88f0c5ff7b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce7c7c06be0846efa867e88f0c5ff7b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce7c7c06be0846efa867e88f0c5ff7b1" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46089 } } }
I20260808 08:25:20.706970 32596 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce7c7c06be0846efa867e88f0c5ff7b1 [sys.catalog]: This master's current role is: LEADER
I20260808 08:25:20.707230 32597 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce7c7c06be0846efa867e88f0c5ff7b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce7c7c06be0846efa867e88f0c5ff7b1. Latest consensus state: current_term: 1 leader_uuid: "ce7c7c06be0846efa867e88f0c5ff7b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce7c7c06be0846efa867e88f0c5ff7b1" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46089 } } }
I20260808 08:25:20.707319 32597 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce7c7c06be0846efa867e88f0c5ff7b1 [sys.catalog]: This master's current role is: LEADER
I20260808 08:25:20.707459 32615 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:25:20.710654 32615 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:25:20.715730 32615 catalog_manager.cc:1347] Generated new cluster ID: 52f3eaa541d64224a572ea0ae015e607
I20260808 08:25:20.715795 32615 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:25:20.720629 32615 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:25:20.721463 32615 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:25:20.726415 32615 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce7c7c06be0846efa867e88f0c5ff7b1: Generated new TSK 0
I20260808 08:25:20.727030 32615 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 08:25:20.749691 32592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:20.737503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47584 (local address 127.22.235.254:46089)
0808 08:25:20.737619 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:20.737624 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:20.737819 (+ 195us) server_negotiation.cc:407] Connection header received
0808 08:25:20.737984 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:20.737987 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:20.738034 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:20.738128 (+ 94us) server_negotiation.cc:226] Negotiated authn=JWT
0808 08:25:20.738363 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:20.738890 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:20.739762 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:20.739980 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:20.740067 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:20.740380 (+ 313us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 08:25:20.749291 (+ 8911us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 08:25:20.749555 (+ 264us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47584: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":44}
W20260808 08:25:20.749776 32634 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:20.737448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46089 (local address 127.0.0.1:47584)
0808 08:25:20.737730 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:20.737742 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:20.737851 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:20.738138 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:20.738141 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:20.738149 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0808 08:25:20.738258 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:20.738264 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:20.739020 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:20.739024 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:20.739641 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:20.739649 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:20.740094 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:20.740098 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:20.740192 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:20.740215 (+ 23us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 08:25:20.749462 (+ 9247us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 08:25:20.749615 (+ 153us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.22.235.254:46089: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":113,"threads_started":1}
W20260808 08:25:20.749776 32592 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47584: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
I20260808 08:25:20.750356 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 32472
2026-08-08T08:25:20Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (289 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (127 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (123 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 08:25:21 dist-test-slave-f7th krb5kdc[32652](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 08:25:21 dist-test-slave-f7th krb5kdc[32652](info): set up 2 sockets
Aug 08 08:25:21 dist-test-slave-f7th krb5kdc[32652](info): commencing operation
krb5kdc: starting...
W20260808 08:25:22.167783 23471 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Aug 08 08:25:22 dist-test-slave-f7th krb5kdc[32652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177522, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T08:25:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:25:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.235.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
I20260808 08:25:22.283420 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:33663
--webserver_interface=127.22.235.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:41535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:33663
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:25:22.394670 32674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:22.394927 32674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:22.394982 32674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:22.398928 32674 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:25:22.398983 32674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:22.399008 32674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:22.399030 32674 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:25:22.399058 32674 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:25:22.404112 32674 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:41535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:33663
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:33663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.235.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32674
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:22.405211 32674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:22.406232 32674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:22.414390 32682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:22.414433 32681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:22.414466 32686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:22.414629 32674 server_base.cc:1034] running on GCE node
I20260808 08:25:22.415283 32674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:22.416318 32674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:22.417481 32674 hybrid_clock.cc:648] HybridClock initialized: now 1786177522417467 us; error 26 us; skew 500 ppm
Aug 08 08:25:22 dist-test-slave-f7th krb5kdc[32652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177522, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:22.422746 32674 init.cc:377] Logged in from keytab as kudu/127.22.235.254@KRBTEST.COM (short username kudu)
I20260808 08:25:22.423624 32674 webserver.cc:460] Webserver started at http://127.22.235.254:32937/ using document root <none> and password file <none>
I20260808 08:25:22.424175 32674 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:22.424244 32674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:22.424453 32674 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:22.426523 32674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "3b906daebc9747c8973f191653f8beda"
format_stamp: "Formatted at 2026-08-08 08:25:22 on dist-test-slave-f7th"
server_key: "5a1d1b05126fca06a1e3304d7eb06eed"
server_key_iv: "16e5de1864868c4567fb994384268501"
server_key_version: "encryptionkey@0"
I20260808 08:25:22.427073 32674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "3b906daebc9747c8973f191653f8beda"
format_stamp: "Formatted at 2026-08-08 08:25:22 on dist-test-slave-f7th"
server_key: "5a1d1b05126fca06a1e3304d7eb06eed"
server_key_iv: "16e5de1864868c4567fb994384268501"
server_key_version: "encryptionkey@0"
I20260808 08:25:22.430692 32674 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260808 08:25:22.433007 32697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:22.434170 32674 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:25:22.434298 32674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "3b906daebc9747c8973f191653f8beda"
format_stamp: "Formatted at 2026-08-08 08:25:22 on dist-test-slave-f7th"
server_key: "5a1d1b05126fca06a1e3304d7eb06eed"
server_key_iv: "16e5de1864868c4567fb994384268501"
server_key_version: "encryptionkey@0"
I20260808 08:25:22.434389 32674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:22.446170 32674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:22.448742 32674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:22.448910 32674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:22.455443 32674 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.254:33663
I20260808 08:25:22.455452 325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.254:33663 every 8 connection(s)
I20260808 08:25:22.456350 32674 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:25:22.458832 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 32674
I20260808 08:25:22.458946 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:25:22.459064 327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:22.459218 23471 external_mini_cluster.cc:1428] Setting key 7037312f3845e02c8bc91a67549a44c7
I20260808 08:25:22.466076 327 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b906daebc9747c8973f191653f8beda: Bootstrap starting.
Aug 08 08:25:22 dist-test-slave-f7th krb5kdc[32652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177522, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:25:22.468540 327 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b906daebc9747c8973f191653f8beda: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:22.469453 327 log.cc:824] T 00000000000000000000000000000000 P 3b906daebc9747c8973f191653f8beda: Log is configured to *not* fsync() on all Append() calls
I20260808 08:25:22.471539 327 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b906daebc9747c8973f191653f8beda: No bootstrap required, opened a new log
I20260808 08:25:22.473459 335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:22.460553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46616 (local address 127.22.235.254:33663)
0808 08:25:22.460994 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:22.461013 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:22.461056 (+ 43us) server_negotiation.cc:407] Connection header received
0808 08:25:22.461781 (+ 725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:22.461806 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:22.462215 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:22.462576 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:22.463231 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:22.464661 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:22.465456 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:22.466133 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:22.466282 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:22.468698 (+ 2416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:22.468704 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:22.468720 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:22.468759 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:22.470398 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:22.470787 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:22.470793 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:22.470798 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:22.470927 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:22.471146 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:22.471149 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:22.471151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:22.471642 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:22.471820 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:22.472024 (+ 204us) server_negotiation.cc:299] Negotiation successful
0808 08:25:22.472299 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":107,"threads_started":1}
I20260808 08:25:22.475157 327 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b906daebc9747c8973f191653f8beda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b906daebc9747c8973f191653f8beda" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 33663 } }
I20260808 08:25:22.475353 327 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b906daebc9747c8973f191653f8beda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:22.475389 327 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b906daebc9747c8973f191653f8beda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b906daebc9747c8973f191653f8beda, State: Initialized, Role: FOLLOWER
I20260808 08:25:22.475823 327 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b906daebc9747c8973f191653f8beda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b906daebc9747c8973f191653f8beda" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 33663 } }
I20260808 08:25:22.475963 327 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b906daebc9747c8973f191653f8beda [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:25:22.476035 327 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b906daebc9747c8973f191653f8beda [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:25:22.476130 327 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b906daebc9747c8973f191653f8beda [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:22.476948 327 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b906daebc9747c8973f191653f8beda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b906daebc9747c8973f191653f8beda" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 33663 } }
I20260808 08:25:22.477252 327 leader_election.cc:302] T 00000000000000000000000000000000 P 3b906daebc9747c8973f191653f8beda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b906daebc9747c8973f191653f8beda; no voters:
I20260808 08:25:22.477509 327 leader_election.cc:288] T 00000000000000000000000000000000 P 3b906daebc9747c8973f191653f8beda [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:25:22.477610 341 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b906daebc9747c8973f191653f8beda [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:25:22.477818 341 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b906daebc9747c8973f191653f8beda [term 1 LEADER]: Becoming Leader. State: Replica: 3b906daebc9747c8973f191653f8beda, State: Running, Role: LEADER
I20260808 08:25:22.478147 341 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b906daebc9747c8973f191653f8beda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b906daebc9747c8973f191653f8beda" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 33663 } }
I20260808 08:25:22.478543 327 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b906daebc9747c8973f191653f8beda [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:25:22.479735 344 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b906daebc9747c8973f191653f8beda [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b906daebc9747c8973f191653f8beda. Latest consensus state: current_term: 1 leader_uuid: "3b906daebc9747c8973f191653f8beda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b906daebc9747c8973f191653f8beda" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 33663 } } }
I20260808 08:25:22.479763 342 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b906daebc9747c8973f191653f8beda [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b906daebc9747c8973f191653f8beda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b906daebc9747c8973f191653f8beda" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 33663 } } }
I20260808 08:25:22.479995 342 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b906daebc9747c8973f191653f8beda [sys.catalog]: This master's current role is: LEADER
I20260808 08:25:22.479966 344 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b906daebc9747c8973f191653f8beda [sys.catalog]: This master's current role is: LEADER
I20260808 08:25:22.480919 358 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:25:22.483632 358 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:25:22.488513 358 catalog_manager.cc:1347] Generated new cluster ID: 0a94173c485d4028b9e96e04f875a14d
I20260808 08:25:22.488574 358 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:25:22.496438 358 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:25:22.497390 358 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:25:22.505438 358 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b906daebc9747c8973f191653f8beda: Generated new TSK 0
I20260808 08:25:22.506110 358 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.235.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab.
Entry for principal kudu/127.22.235.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab.
I20260808 08:25:22.532332 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.193:0
--local_ip_for_outbound_sockets=127.22.235.193
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:33663
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab
--principal=kudu/127.22.235.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:41535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:25:22.643268 373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:22.643510 373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:22.643556 373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:22.647511 373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:22.647563 373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:22.647631 373 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.193
I20260808 08:25:22.652753 373 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:41535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab
--principal=kudu/127.22.235.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:33663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.373
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.193
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:22.653738 373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:22.654721 373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:22.663748 378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:22.663754 379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:22.663842 382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:22.663918 373 server_base.cc:1034] running on GCE node
I20260808 08:25:22.664381 373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:22.665014 373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:22.666167 373 hybrid_clock.cc:648] HybridClock initialized: now 1786177522666151 us; error 28 us; skew 500 ppm
Aug 08 08:25:22 dist-test-slave-f7th krb5kdc[32652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177522, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:22.668903 373 init.cc:377] Logged in from keytab as kudu/127.22.235.193@KRBTEST.COM (short username kudu)
I20260808 08:25:22.669734 373 webserver.cc:460] Webserver started at http://127.22.235.193:40901/ using document root <none> and password file <none>
I20260808 08:25:22.670267 373 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:22.670346 373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:22.670550 373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:22.672494 373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-0/data/instance:
uuid: "db72665aba284010943a686bd73fb3af"
format_stamp: "Formatted at 2026-08-08 08:25:22 on dist-test-slave-f7th"
server_key: "e0cf9b4efd910d8080b561ba940a698a"
server_key_iv: "96f79b17208b96b4894639240d7cd0b9"
server_key_version: "encryptionkey@0"
I20260808 08:25:22.673058 373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance:
uuid: "db72665aba284010943a686bd73fb3af"
format_stamp: "Formatted at 2026-08-08 08:25:22 on dist-test-slave-f7th"
server_key: "e0cf9b4efd910d8080b561ba940a698a"
server_key_iv: "96f79b17208b96b4894639240d7cd0b9"
server_key_version: "encryptionkey@0"
I20260808 08:25:22.676560 373 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 08:25:22.678925 391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:22.680173 373 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 08:25:22.680276 373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-0/wal
uuid: "db72665aba284010943a686bd73fb3af"
format_stamp: "Formatted at 2026-08-08 08:25:22 on dist-test-slave-f7th"
server_key: "e0cf9b4efd910d8080b561ba940a698a"
server_key_iv: "96f79b17208b96b4894639240d7cd0b9"
server_key_version: "encryptionkey@0"
I20260808 08:25:22.680365 373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:22.688983 373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:22.691365 373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:22.691532 373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:22.692039 373 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:25:22.692821 373 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:25:22.692867 373 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:22.692917 373 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:25:22.692945 373 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:22.701056 373 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.193:33165
I20260808 08:25:22.701105 578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.193:33165 every 8 connection(s)
I20260808 08:25:22.701875 373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
I20260808 08:25:22.708313 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 373
I20260808 08:25:22.708428 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance
I20260808 08:25:22.708756 23471 external_mini_cluster.cc:1428] Setting key cae5b164d7bb27aaaa9f4b90be2043a0
Aug 08 08:25:22 dist-test-slave-f7th krb5kdc[32652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177522, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:25:22.715193 335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:22.703535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:56335 (local address 127.22.235.254:33663)
0808 08:25:22.703756 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:22.703762 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:22.704780 (+ 1018us) server_negotiation.cc:407] Connection header received
0808 08:25:22.706109 (+ 1329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:22.706112 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:22.706161 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:22.706230 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:22.707293 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:22.707814 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:22.709260 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:22.709559 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:22.709651 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:22.711928 (+ 2277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:22.711933 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:22.711935 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:22.711961 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:22.713294 (+ 1333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:22.713739 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:22.713742 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:22.713744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:22.713821 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:22.714123 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:22.714127 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:22.714128 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:22.714300 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:22.714386 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:22.714892 (+ 506us) server_negotiation.cc:299] Negotiation successful
0808 08:25:22.715023 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":111}
I20260808 08:25:22.715963 582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:22.704131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:33663 (local address 127.22.235.193:56335)
0808 08:25:22.704600 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:22.704678 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:22.705894 (+ 1216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:22.706369 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:22.706375 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:22.706704 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:22.707179 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:22.707192 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:22.707897 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:22.707899 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:22.709141 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:22.709153 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:22.709701 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:22.709706 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:22.709874 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:22.710442 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:22.710458 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:22.711780 (+ 1322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:22.713392 (+ 1612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:22.713398 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:22.713411 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:22.713649 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:22.713898 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:22.713901 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:22.713903 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:22.714038 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:22.714395 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:22.714402 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:22.714628 (+ 226us) client_negotiation.cc:769] Sending connection context
0808 08:25:22.714810 (+ 182us) client_negotiation.cc:240] Negotiation successful
0808 08:25:22.715113 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":125,"threads_started":1}
I20260808 08:25:22.717072 579 heartbeater.cc:347] Connected to a master server at 127.22.235.254:33663
I20260808 08:25:22.717334 579 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:22.717905 579 heartbeater.cc:511] Master 127.22.235.254:33663 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.22.235.194@KRBTEST.COM; defaulting to no policy
I20260808 08:25:22.719463 32732 ts_manager.cc:194] Registered new tserver with Master: db72665aba284010943a686bd73fb3af (127.22.235.193:33165)
I20260808 08:25:22.720553 32732 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.193@KRBTEST.COM'} at 127.22.235.193:56335
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab.
Entry for principal kudu/127.22.235.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab.
I20260808 08:25:22.733814 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.194:0
--local_ip_for_outbound_sockets=127.22.235.194
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:33663
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab
--principal=kudu/127.22.235.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:41535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:25:22.843762 587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:22.844020 587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:22.844061 587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:22.848363 587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:22.848428 587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:22.848510 587 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.194
I20260808 08:25:22.853873 587 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:41535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab
--principal=kudu/127.22.235.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:33663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.587
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.194
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:22.854888 587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:22.855901 587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:22.865269 604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:22.865295 599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:22.865274 598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:22.865486 587 server_base.cc:1034] running on GCE node
I20260808 08:25:22.865957 587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:22.866559 587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:22.867719 587 hybrid_clock.cc:648] HybridClock initialized: now 1786177522867700 us; error 32 us; skew 500 ppm
Aug 08 08:25:22 dist-test-slave-f7th krb5kdc[32652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177522, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:22.870327 587 init.cc:377] Logged in from keytab as kudu/127.22.235.194@KRBTEST.COM (short username kudu)
I20260808 08:25:22.871121 587 webserver.cc:460] Webserver started at http://127.22.235.194:40131/ using document root <none> and password file <none>
I20260808 08:25:22.871647 587 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:22.871706 587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:22.871877 587 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:22.873871 587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-1/data/instance:
uuid: "98152a1b7dda41cb9015ee047c2deda7"
format_stamp: "Formatted at 2026-08-08 08:25:22 on dist-test-slave-f7th"
server_key: "dba2a21b78ed113a17cfb8dccd7e85b1"
server_key_iv: "2fbb94a4ba11a4ce17397229173f2c09"
server_key_version: "encryptionkey@0"
I20260808 08:25:22.874373 587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance:
uuid: "98152a1b7dda41cb9015ee047c2deda7"
format_stamp: "Formatted at 2026-08-08 08:25:22 on dist-test-slave-f7th"
server_key: "dba2a21b78ed113a17cfb8dccd7e85b1"
server_key_iv: "2fbb94a4ba11a4ce17397229173f2c09"
server_key_version: "encryptionkey@0"
I20260808 08:25:22.877919 587 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 08:25:22.880402 612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:22.881570 587 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:25:22.881676 587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-1/wal
uuid: "98152a1b7dda41cb9015ee047c2deda7"
format_stamp: "Formatted at 2026-08-08 08:25:22 on dist-test-slave-f7th"
server_key: "dba2a21b78ed113a17cfb8dccd7e85b1"
server_key_iv: "2fbb94a4ba11a4ce17397229173f2c09"
server_key_version: "encryptionkey@0"
I20260808 08:25:22.881773 587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:22.904192 587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:22.907030 587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:22.907197 587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:22.907773 587 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:25:22.908592 587 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:25:22.908702 587 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:22.908814 587 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:25:22.908846 587 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:22.916976 587 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.194:34805
I20260808 08:25:22.917034 811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.194:34805 every 8 connection(s)
I20260808 08:25:22.917819 587 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
I20260808 08:25:22.919296 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 587
I20260808 08:25:22.919411 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance
I20260808 08:25:22.919724 23471 external_mini_cluster.cc:1428] Setting key f188883152c73b103de592f6e754af9b
Aug 08 08:25:22 dist-test-slave-f7th krb5kdc[32652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177522, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.194@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.235.195@KRBTEST.COM; defaulting to no policy
I20260808 08:25:22.930989 335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:22.919525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:50485 (local address 127.22.235.254:33663)
0808 08:25:22.919682 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:22.919687 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:22.920526 (+ 839us) server_negotiation.cc:407] Connection header received
0808 08:25:22.921938 (+ 1412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:22.921941 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:22.922005 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:22.922084 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:22.923217 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:22.923755 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:22.924889 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:22.925107 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:22.925195 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:22.927633 (+ 2438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:22.927636 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:22.927638 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:22.927662 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:22.928986 (+ 1324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:22.929584 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:22.929586 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:22.929587 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:22.929674 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:22.930005 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:22.930007 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:22.930008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:22.930194 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:22.930265 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:22.930753 (+ 488us) server_negotiation.cc:299] Negotiation successful
0808 08:25:22.930868 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 08:25:22.931720 815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:22.919920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:33663 (local address 127.22.235.194:50485)
0808 08:25:22.920383 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:22.920432 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:22.921723 (+ 1291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:22.922238 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:22.922246 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:22.922610 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:22.923075 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:22.923089 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:22.923873 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:22.923877 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:22.924778 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:22.924789 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:22.925237 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:22.925242 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:22.925410 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:22.925909 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:22.925926 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:22.927450 (+ 1524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:22.929101 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:22.929107 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:22.929122 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:22.929495 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:22.929746 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:22.929749 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:22.929751 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:22.929929 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:22.930268 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:22.930273 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:22.930508 (+ 235us) client_negotiation.cc:769] Sending connection context
0808 08:25:22.930701 (+ 193us) client_negotiation.cc:240] Negotiation successful
0808 08:25:22.930948 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":116,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.195@KRBTEST.COM" created.
I20260808 08:25:22.932900 812 heartbeater.cc:347] Connected to a master server at 127.22.235.254:33663
I20260808 08:25:22.933168 812 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:22.933660 812 heartbeater.cc:511] Master 127.22.235.254:33663 requested a full tablet report, sending...
I20260808 08:25:22.934779 32732 ts_manager.cc:194] Registered new tserver with Master: 98152a1b7dda41cb9015ee047c2deda7 (127.22.235.194:34805)
I20260808 08:25:22.935329 32732 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.194@KRBTEST.COM'} at 127.22.235.194:50485
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab.
Entry for principal kudu/127.22.235.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab.
I20260808 08:25:22.947639 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.195:0
--local_ip_for_outbound_sockets=127.22.235.195
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:33663
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab
--principal=kudu/127.22.235.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:41535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:25:23.059780 818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:23.060039 818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:23.060096 818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:23.064221 818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:23.064283 818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:23.064365 818 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.195
I20260808 08:25:23.069479 818 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:41535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab
--principal=kudu/127.22.235.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:33663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.195
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:23.070465 818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:23.071484 818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:23.081611 827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:23.081671 829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:23.081671 834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:23.082037 818 server_base.cc:1034] running on GCE node
I20260808 08:25:23.082393 818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:23.082984 818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:23.084149 818 hybrid_clock.cc:648] HybridClock initialized: now 1786177523084118 us; error 39 us; skew 500 ppm
Aug 08 08:25:23 dist-test-slave-f7th krb5kdc[32652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177523, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:23.086937 818 init.cc:377] Logged in from keytab as kudu/127.22.235.195@KRBTEST.COM (short username kudu)
I20260808 08:25:23.087808 818 webserver.cc:460] Webserver started at http://127.22.235.195:40093/ using document root <none> and password file <none>
I20260808 08:25:23.088340 818 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:23.088407 818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:23.088696 818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:23.090696 818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-2/data/instance:
uuid: "d2bb63c7101f457d91e67d212ec18996"
format_stamp: "Formatted at 2026-08-08 08:25:23 on dist-test-slave-f7th"
server_key: "6e50b6c78a03181dfee53b54d7e1f792"
server_key_iv: "6a3bb310d591bdb38b2347bdafcf8d17"
server_key_version: "encryptionkey@0"
I20260808 08:25:23.091233 818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance:
uuid: "d2bb63c7101f457d91e67d212ec18996"
format_stamp: "Formatted at 2026-08-08 08:25:23 on dist-test-slave-f7th"
server_key: "6e50b6c78a03181dfee53b54d7e1f792"
server_key_iv: "6a3bb310d591bdb38b2347bdafcf8d17"
server_key_version: "encryptionkey@0"
I20260808 08:25:23.094816 818 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 08:25:23.097146 845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:23.098313 818 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:25:23.098425 818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-2/wal
uuid: "d2bb63c7101f457d91e67d212ec18996"
format_stamp: "Formatted at 2026-08-08 08:25:23 on dist-test-slave-f7th"
server_key: "6e50b6c78a03181dfee53b54d7e1f792"
server_key_iv: "6a3bb310d591bdb38b2347bdafcf8d17"
server_key_version: "encryptionkey@0"
I20260808 08:25:23.098526 818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:23.115634 818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:23.118676 818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:23.118863 818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:23.119431 818 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:25:23.120249 818 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:25:23.120308 818 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:23.120364 818 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:25:23.120393 818 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:23.129072 818 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.195:43125
I20260808 08:25:23.129129 1045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.195:43125 every 8 connection(s)
I20260808 08:25:23.129911 818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
I20260808 08:25:23.133535 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 818
I20260808 08:25:23.133652 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance
I20260808 08:25:23.133930 23471 external_mini_cluster.cc:1428] Setting key 447a9ceda0293237d4cf117efdcbddb8
Aug 08 08:25:23 dist-test-slave-f7th krb5kdc[32652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177523, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.195@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:25:23.143021 335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:23.131614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:38003 (local address 127.22.235.254:33663)
0808 08:25:23.131731 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:23.131735 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:23.132501 (+ 766us) server_negotiation.cc:407] Connection header received
0808 08:25:23.134101 (+ 1600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:23.134104 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:23.134156 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:23.134219 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:23.135625 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:23.136241 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:23.137405 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:23.137624 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:23.137712 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:23.139953 (+ 2241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:23.139957 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:23.139959 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:23.139983 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:23.141164 (+ 1181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:23.141621 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:23.141624 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:23.141625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:23.141701 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:23.142001 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:23.142005 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:23.142006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:23.142177 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:23.142260 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:23.142733 (+ 473us) server_negotiation.cc:299] Negotiation successful
0808 08:25:23.142854 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 08:25:23.143733 1050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:23.131921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:33663 (local address 127.22.235.195:38003)
0808 08:25:23.132339 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:23.132379 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:23.133867 (+ 1488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:23.134390 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:23.134396 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:23.134835 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:23.135431 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:23.135470 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:23.136359 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:23.136362 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:23.137296 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:23.137306 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:23.137758 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:23.137762 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:23.137936 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:23.138426 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:23.138442 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:23.139806 (+ 1364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:23.141267 (+ 1461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:23.141273 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:23.141286 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:23.141530 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:23.141781 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:23.141784 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:23.141786 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:23.141909 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:23.142275 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:23.142282 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:23.142511 (+ 229us) client_negotiation.cc:769] Sending connection context
0808 08:25:23.142672 (+ 161us) client_negotiation.cc:240] Negotiation successful
0808 08:25:23.142969 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":110,"threads_started":1}
I20260808 08:25:23.144834 1047 heartbeater.cc:347] Connected to a master server at 127.22.235.254:33663
I20260808 08:25:23.145047 1047 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:23.145483 1047 heartbeater.cc:511] Master 127.22.235.254:33663 requested a full tablet report, sending...
I20260808 08:25:23.146493 32732 ts_manager.cc:194] Registered new tserver with Master: d2bb63c7101f457d91e67d212ec18996 (127.22.235.195:43125)
I20260808 08:25:23.147162 32732 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.195@KRBTEST.COM'} at 127.22.235.195:38003
I20260808 08:25:23.147531 23471 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/krb5kdc/test-admin.keytab.
Aug 08 08:25:23 dist-test-slave-f7th krb5kdc[32652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:23.162685 23471 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260808 08:25:23.165565 1056 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:23.163990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51518 (local address 127.22.235.193:33165)
0808 08:25:23.164337 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:23.164348 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:23.164369 (+ 21us) server_negotiation.cc:407] Connection header received
0808 08:25:23.164417 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:23.164422 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:23.164647 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:23.164785 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:23.165379 (+ 594us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51518: BlockingRecv error: recv got EOF from 127.0.0.1:51518 (error 108)
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":130,"threads_started":1}
W20260808 08:25:23.165619 23471 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/krb5kdc/krb5cc' of type FILE
Aug 08 08:25:23 dist-test-slave-f7th krb5kdc[32652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:23.166579 23471 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 08:25:23 dist-test-slave-f7th krb5kdc[32652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:23.167464 23471 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 08:25:23 dist-test-slave-f7th krb5kdc[32652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.235.193@KRBTEST.COM
I20260808 08:25:23.176522 1056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:23.168266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51530 (local address 127.22.235.193:33165)
0808 08:25:23.168378 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:23.168382 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:23.168435 (+ 53us) server_negotiation.cc:407] Connection header received
0808 08:25:23.168553 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:23.168556 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:23.168603 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:23.168701 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:23.169186 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:23.170165 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:23.171037 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:23.171414 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:23.171504 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:23.173699 (+ 2195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:23.173709 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:23.173723 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:23.173747 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:23.175083 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:23.175453 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:23.175459 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:23.175461 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:23.175540 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:23.175731 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:23.175733 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:23.175735 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:23.175957 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:23.176130 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:23.176260 (+ 130us) server_negotiation.cc:299] Negotiation successful
0808 08:25:23.176378 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260808 08:25:23.180054 1056 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:23.179230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51534 (local address 127.22.235.193:33165)
0808 08:25:23.179355 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:23.179359 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:23.179430 (+ 71us) server_negotiation.cc:407] Connection header received
0808 08:25:23.179526 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:23.179528 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:23.179563 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:23.179620 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:23.179948 (+ 328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51534: BlockingRecv error: recv got EOF from 127.0.0.1:51534 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260808 08:25:23.180150 23471 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/krb5kdc/krb5cc' of type FILE
Aug 08 08:25:23 dist-test-slave-f7th krb5kdc[32652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:23.181123 23471 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 08:25:23 dist-test-slave-f7th krb5kdc[32652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:23.181905 23471 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 08:25:23 dist-test-slave-f7th krb5kdc[32652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.235.193@KRBTEST.COM
I20260808 08:25:23.189440 1056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:23.182604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51548 (local address 127.22.235.193:33165)
0808 08:25:23.182704 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:23.182708 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:23.182786 (+ 78us) server_negotiation.cc:407] Connection header received
0808 08:25:23.182860 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:23.182861 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:23.182891 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:23.182952 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:23.183342 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:23.183927 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:23.184727 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:23.185009 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:23.185088 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:23.187040 (+ 1952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:23.187044 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:23.187045 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:23.187067 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:23.188268 (+ 1201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:23.188578 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:23.188580 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:23.188582 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:23.188680 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:23.188876 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:23.188878 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:23.188879 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:23.189032 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:23.189118 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:23.189263 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 08:25:23.189345 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260808 08:25:23.191533 1056 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:23.190831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51562 (local address 127.22.235.193:33165)
0808 08:25:23.190944 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:23.190947 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:23.191000 (+ 53us) server_negotiation.cc:407] Connection header received
0808 08:25:23.191078 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:23.191080 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:23.191109 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:23.191164 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:23.191456 (+ 292us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51562: BlockingRecv error: recv got EOF from 127.0.0.1:51562 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260808 08:25:23.191661 23471 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786177493204855-23471-0/krb5kdc/krb5cc' of type FILE
Aug 08 08:25:23 dist-test-slave-f7th krb5kdc[32652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:23.192754 23471 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 08:25:23 dist-test-slave-f7th krb5kdc[32652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:23.193580 23471 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 08:25:23 dist-test-slave-f7th krb5kdc[32652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.235.193@KRBTEST.COM
I20260808 08:25:23.200376 1056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:23.194306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51566 (local address 127.22.235.193:33165)
0808 08:25:23.194416 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:23.194419 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:23.194466 (+ 47us) server_negotiation.cc:407] Connection header received
0808 08:25:23.194566 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:23.194568 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:23.194599 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:23.194649 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:23.195028 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:23.195461 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:23.196109 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:23.196296 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:23.196357 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:23.198213 (+ 1856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:23.198216 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:23.198217 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:23.198236 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:23.199250 (+ 1014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:23.199583 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:23.199585 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:23.199587 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:23.199645 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:23.199815 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:23.199816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:23.199817 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:23.199939 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:23.200029 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:23.200171 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 08:25:23.200247 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 08:25:23.201258 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 373
I20260808 08:25:23.215807 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 587
I20260808 08:25:23.227267 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 818
I20260808 08:25:23.241364 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 32674
2026-08-08T08:25:23Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2242 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 08:25:23 dist-test-slave-f7th krb5kdc[1076](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 08:25:23 dist-test-slave-f7th krb5kdc[1076](info): set up 2 sockets
Aug 08 08:25:23 dist-test-slave-f7th krb5kdc[1076](info): commencing operation
krb5kdc: starting...
W20260808 08:25:24.404788 23471 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Aug 08 08:25:24 dist-test-slave-f7th krb5kdc[1076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177524, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T08:25:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:25:24Z Disabled control of system clock
WARNING: no policy specified for oryx/127.22.235.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.235.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.235.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.254.keytab.
Entry for principal oryx/127.22.235.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.254.keytab.
I20260808 08:25:24.523591 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:46527
--webserver_interface=127.22.235.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.254.keytab
--principal=oryx/127.22.235.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:36355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:46527
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:25:24.634125 1099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:24.634383 1099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:24.634438 1099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:24.638319 1099 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:25:24.638372 1099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:24.638399 1099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:24.638420 1099 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:25:24.638449 1099 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:25:24.643879 1099 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:36355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:46527
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.254.keytab
--principal=oryx/127.22.235.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:46527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.235.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1099
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:24.644924 1099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:24.645963 1099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:24.654135 1108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:24.654163 1107 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:24.654294 1110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:24.654282 1099 server_base.cc:1034] running on GCE node
I20260808 08:25:24.654948 1099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:24.655972 1099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:24.657132 1099 hybrid_clock.cc:648] HybridClock initialized: now 1786177524657105 us; error 35 us; skew 500 ppm
Aug 08 08:25:24 dist-test-slave-f7th krb5kdc[1076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177524, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.235.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:24.659828 1099 init.cc:377] Logged in from keytab as oryx/127.22.235.254@KRBTEST.COM (short username oryx)
I20260808 08:25:24.660699 1099 webserver.cc:460] Webserver started at http://127.22.235.254:34397/ using document root <none> and password file <none>
I20260808 08:25:24.661232 1099 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:24.661300 1099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:24.661507 1099 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:24.663486 1099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "00ae12b3a32549bb9d8a3092fa6e0139"
format_stamp: "Formatted at 2026-08-08 08:25:24 on dist-test-slave-f7th"
server_key: "ed69c0d6346d94cf853fd5473ad6cb95"
server_key_iv: "5f05e0295fba60a5ac73b234b0336207"
server_key_version: "encryptionkey@0"
I20260808 08:25:24.664026 1099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "00ae12b3a32549bb9d8a3092fa6e0139"
format_stamp: "Formatted at 2026-08-08 08:25:24 on dist-test-slave-f7th"
server_key: "ed69c0d6346d94cf853fd5473ad6cb95"
server_key_iv: "5f05e0295fba60a5ac73b234b0336207"
server_key_version: "encryptionkey@0"
I20260808 08:25:24.667574 1099 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 08:25:24.669873 1125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:24.671023 1099 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 08:25:24.671141 1099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "00ae12b3a32549bb9d8a3092fa6e0139"
format_stamp: "Formatted at 2026-08-08 08:25:24 on dist-test-slave-f7th"
server_key: "ed69c0d6346d94cf853fd5473ad6cb95"
server_key_iv: "5f05e0295fba60a5ac73b234b0336207"
server_key_version: "encryptionkey@0"
I20260808 08:25:24.671231 1099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:24.683779 1099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:24.686755 1099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:24.686939 1099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:24.693411 1099 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.254:46527
I20260808 08:25:24.693691 1209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.254:46527 every 8 connection(s)
I20260808 08:25:24.694393 1099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:25:24.697364 1210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:24.699245 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 1099
I20260808 08:25:24.699369 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:25:24.699684 23471 external_mini_cluster.cc:1428] Setting key c743eafc1e47bee5af15ff6d10fce1bf
I20260808 08:25:24.703641 1210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139: Bootstrap starting.
I20260808 08:25:24.706053 1210 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:24.706807 1210 log.cc:824] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139: Log is configured to *not* fsync() on all Append() calls
Aug 08 08:25:24 dist-test-slave-f7th krb5kdc[1076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177524, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM
I20260808 08:25:24.708925 1210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139: No bootstrap required, opened a new log
I20260808 08:25:24.712641 1210 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } }
I20260808 08:25:24.712857 1210 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:24.712914 1210 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00ae12b3a32549bb9d8a3092fa6e0139, State: Initialized, Role: FOLLOWER
I20260808 08:25:24.713343 1210 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } }
I20260808 08:25:24.713394 1213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:24.701118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44706 (local address 127.22.235.254:46527)
0808 08:25:24.701533 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:24.701548 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:24.701582 (+ 34us) server_negotiation.cc:407] Connection header received
0808 08:25:24.702214 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:24.702233 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:24.702557 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:24.702854 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:24.703830 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:24.704903 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:24.706040 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:24.706542 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:24.706663 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:24.709194 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:24.709199 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:24.709212 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:24.709238 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:24.710747 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:24.711193 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:24.711197 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:24.711201 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:24.711304 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:24.711558 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:24.711559 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:24.711560 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:24.711908 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:24.712064 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:24.712315 (+ 251us) server_negotiation.cc:299] Negotiation successful
0808 08:25:24.712586 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":123,"threads_started":1}
I20260808 08:25:24.713485 1210 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:25:24.713547 1210 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:25:24.713662 1210 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:24.714643 1210 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } }
I20260808 08:25:24.714949 1210 leader_election.cc:302] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00ae12b3a32549bb9d8a3092fa6e0139; no voters:
I20260808 08:25:24.715234 1210 leader_election.cc:288] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:25:24.715322 1215 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:25:24.715811 1215 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 [term 1 LEADER]: Becoming Leader. State: Replica: 00ae12b3a32549bb9d8a3092fa6e0139, State: Running, Role: LEADER
I20260808 08:25:24.716154 1215 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } }
I20260808 08:25:24.716460 1210 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:25:24.717854 1218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00ae12b3a32549bb9d8a3092fa6e0139. Latest consensus state: current_term: 1 leader_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } }
I20260808 08:25:24.717865 1216 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } }
I20260808 08:25:24.717999 1216 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 [sys.catalog]: This master's current role is: LEADER
I20260808 08:25:24.717999 1218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 [sys.catalog]: This master's current role is: LEADER
I20260808 08:25:24.718506 1230 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:25:24.721585 1230 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:25:24.726475 1230 catalog_manager.cc:1347] Generated new cluster ID: aafd38e6ed664f44b830a3f06aac46a6
I20260808 08:25:24.726591 1230 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:25:24.736907 1230 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:25:24.738014 1230 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:25:24.746322 1230 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139: Generated new TSK 0
I20260808 08:25:24.746968 1230 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.22.235.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.235.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.235.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.193.keytab.
Entry for principal oryx/127.22.235.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.193.keytab.
I20260808 08:25:24.775278 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.193:0
--local_ip_for_outbound_sockets=127.22.235.193
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:46527
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.193.keytab
--principal=oryx/127.22.235.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:36355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:25:24.888003 1251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:24.888232 1251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:24.888274 1251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:24.892194 1251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:24.892247 1251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:24.892307 1251 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.193
I20260808 08:25:24.897503 1251 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:36355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.193.keytab
--principal=oryx/127.22.235.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:46527
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1251
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.193
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:24.898406 1251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:24.899314 1251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:24.908527 1260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:24.908589 1263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:24.908672 1251 server_base.cc:1034] running on GCE node
W20260808 08:25:24.908540 1261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:24.909193 1251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:24.909775 1251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:24.910966 1251 hybrid_clock.cc:648] HybridClock initialized: now 1786177524910939 us; error 38 us; skew 500 ppm
Aug 08 08:25:24 dist-test-slave-f7th krb5kdc[1076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177524, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.235.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:24.913692 1251 init.cc:377] Logged in from keytab as oryx/127.22.235.193@KRBTEST.COM (short username oryx)
I20260808 08:25:24.914551 1251 webserver.cc:460] Webserver started at http://127.22.235.193:36801/ using document root <none> and password file <none>
I20260808 08:25:24.915081 1251 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:24.915143 1251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:24.915340 1251 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:24.917405 1251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-0/data/instance:
uuid: "11de37660d0946beb33ae725e791d907"
format_stamp: "Formatted at 2026-08-08 08:25:24 on dist-test-slave-f7th"
server_key: "4d6a72b58c789a36984279c4545d821e"
server_key_iv: "bdb93fedea4f81c3e18169357cc7d0be"
server_key_version: "encryptionkey@0"
I20260808 08:25:24.917971 1251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance:
uuid: "11de37660d0946beb33ae725e791d907"
format_stamp: "Formatted at 2026-08-08 08:25:24 on dist-test-slave-f7th"
server_key: "4d6a72b58c789a36984279c4545d821e"
server_key_iv: "bdb93fedea4f81c3e18169357cc7d0be"
server_key_version: "encryptionkey@0"
I20260808 08:25:24.921587 1251 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 08:25:24.923900 1281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:24.925102 1251 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 08:25:24.925204 1251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-0/wal
uuid: "11de37660d0946beb33ae725e791d907"
format_stamp: "Formatted at 2026-08-08 08:25:24 on dist-test-slave-f7th"
server_key: "4d6a72b58c789a36984279c4545d821e"
server_key_iv: "bdb93fedea4f81c3e18169357cc7d0be"
server_key_version: "encryptionkey@0"
I20260808 08:25:24.925298 1251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:24.938995 1251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:24.941998 1251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:24.942166 1251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:24.942721 1251 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:25:24.943526 1251 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:25:24.943573 1251 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:24.943615 1251 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:25:24.943646 1251 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:24.951858 1251 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.193:39011
I20260808 08:25:24.951877 1476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.193:39011 every 8 connection(s)
I20260808 08:25:24.952725 1251 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
Aug 08 08:25:24 dist-test-slave-f7th krb5kdc[1076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177524, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.235.193@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM
I20260808 08:25:24.961796 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 1251
I20260808 08:25:24.961902 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance
I20260808 08:25:24.962200 23471 external_mini_cluster.cc:1428] Setting key 6740589fa652b01cb26853ee7e77a834
I20260808 08:25:24.965945 1213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:24.954479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:34843 (local address 127.22.235.254:46527)
0808 08:25:24.954612 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:24.954615 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:24.955385 (+ 770us) server_negotiation.cc:407] Connection header received
0808 08:25:24.956762 (+ 1377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:24.956765 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:24.956824 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:24.956901 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:24.958066 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:24.958605 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:24.959706 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:24.959938 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:24.960028 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:24.962410 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:24.962414 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:24.962416 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:24.962446 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:24.963973 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:24.964457 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:24.964460 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:24.964462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:24.964566 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:24.964921 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:24.964924 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:24.964925 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:24.965137 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:24.965228 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:24.965708 (+ 480us) server_negotiation.cc:299] Negotiation successful
0808 08:25:24.965826 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 08:25:24.966984 1481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:24.954802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46527 (local address 127.22.235.193:34843)
0808 08:25:24.955246 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:24.955287 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:24.956527 (+ 1240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:24.957063 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:24.957070 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:24.957407 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:24.957916 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:24.957931 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:24.958728 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:24.958731 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:24.959581 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:24.959590 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:24.960075 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:24.960079 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:24.960236 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:24.960775 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:24.960792 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:24.962200 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:24.964107 (+ 1907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:24.964112 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:24.964123 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:24.964365 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:24.964677 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:24.964680 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:24.964681 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:24.964816 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:24.965237 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:24.965242 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:24.965462 (+ 220us) client_negotiation.cc:769] Sending connection context
0808 08:25:24.965649 (+ 187us) client_negotiation.cc:240] Negotiation successful
0808 08:25:24.965977 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":121,"threads_started":1}
I20260808 08:25:24.968164 1479 heartbeater.cc:347] Connected to a master server at 127.22.235.254:46527
I20260808 08:25:24.968393 1479 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:24.968945 1479 heartbeater.cc:511] Master 127.22.235.254:46527 requested a full tablet report, sending...
I20260808 08:25:24.970471 1158 ts_manager.cc:194] Registered new tserver with Master: 11de37660d0946beb33ae725e791d907 (127.22.235.193:39011)
I20260808 08:25:24.971609 1158 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.235.193@KRBTEST.COM'} at 127.22.235.193:34843
WARNING: no policy specified for oryx/127.22.235.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.235.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.235.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.194.keytab.
Entry for principal oryx/127.22.235.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.194.keytab.
I20260808 08:25:24.989380 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.194:0
--local_ip_for_outbound_sockets=127.22.235.194
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:46527
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.194.keytab
--principal=oryx/127.22.235.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:36355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:25:25.102100 1485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:25.102334 1485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:25.102378 1485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:25.106289 1485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:25.106343 1485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:25.106410 1485 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.194
I20260808 08:25:25.111498 1485 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:36355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.194.keytab
--principal=oryx/127.22.235.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:46527
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.194
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:25.112473 1485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:25.113469 1485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 08:25:25.122956 1485 server_base.cc:1034] running on GCE node
W20260808 08:25:25.122906 1494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:25.122906 1499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:25.122916 1497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:25.123564 1485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:25.124172 1485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:25.125327 1485 hybrid_clock.cc:648] HybridClock initialized: now 1786177525125310 us; error 29 us; skew 500 ppm
Aug 08 08:25:25 dist-test-slave-f7th krb5kdc[1076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177525, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.235.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:25.128151 1485 init.cc:377] Logged in from keytab as oryx/127.22.235.194@KRBTEST.COM (short username oryx)
I20260808 08:25:25.129073 1485 webserver.cc:460] Webserver started at http://127.22.235.194:36631/ using document root <none> and password file <none>
I20260808 08:25:25.129601 1485 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:25.129688 1485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:25.129870 1485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:25.131805 1485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-1/data/instance:
uuid: "6b3a62b7e6a249d197410707a5bc428a"
format_stamp: "Formatted at 2026-08-08 08:25:25 on dist-test-slave-f7th"
server_key: "a94874ade4d471b354557b00af091e93"
server_key_iv: "ab19031d75155b0ce583ae6bc7d4db7c"
server_key_version: "encryptionkey@0"
I20260808 08:25:25.132308 1485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance:
uuid: "6b3a62b7e6a249d197410707a5bc428a"
format_stamp: "Formatted at 2026-08-08 08:25:25 on dist-test-slave-f7th"
server_key: "a94874ade4d471b354557b00af091e93"
server_key_iv: "ab19031d75155b0ce583ae6bc7d4db7c"
server_key_version: "encryptionkey@0"
I20260808 08:25:25.135880 1485 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 08:25:25.138195 1508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:25.139331 1485 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260808 08:25:25.139441 1485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-1/wal
uuid: "6b3a62b7e6a249d197410707a5bc428a"
format_stamp: "Formatted at 2026-08-08 08:25:25 on dist-test-slave-f7th"
server_key: "a94874ade4d471b354557b00af091e93"
server_key_iv: "ab19031d75155b0ce583ae6bc7d4db7c"
server_key_version: "encryptionkey@0"
I20260808 08:25:25.139550 1485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:25.150786 1485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:25.153434 1485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:25.153594 1485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:25.154139 1485 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:25:25.154930 1485 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:25:25.154996 1485 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:25.155058 1485 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:25:25.155089 1485 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:25.163304 1485 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.194:39211
I20260808 08:25:25.163326 1694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.194:39211 every 8 connection(s)
I20260808 08:25:25.164160 1485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
I20260808 08:25:25.165256 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 1485
I20260808 08:25:25.165349 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance
I20260808 08:25:25.165616 23471 external_mini_cluster.cc:1428] Setting key 83625e87cefe5b997e7f512a852334b9
Aug 08 08:25:25 dist-test-slave-f7th krb5kdc[1076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177525, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.235.194@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM
WARNING: no policy specified for oryx/127.22.235.195@KRBTEST.COM; defaulting to no policy
I20260808 08:25:25.178229 1213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:25.165985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:45491 (local address 127.22.235.254:46527)
0808 08:25:25.166134 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:25.166138 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:25.167043 (+ 905us) server_negotiation.cc:407] Connection header received
0808 08:25:25.168485 (+ 1442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:25.168489 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:25.168544 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:25.168636 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:25.170149 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:25.170682 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:25.171841 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:25.172068 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:25.172162 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:25.174589 (+ 2427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:25.174592 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:25.174594 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:25.174619 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:25.176038 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:25.176521 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:25.176525 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:25.176526 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:25.176605 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:25.177019 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:25.177023 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:25.177024 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:25.177256 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:25.177351 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:25.177856 (+ 505us) server_negotiation.cc:299] Negotiation successful
0808 08:25:25.178030 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.235.195@KRBTEST.COM" created.
I20260808 08:25:25.179040 1698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:25.166392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46527 (local address 127.22.235.194:45491)
0808 08:25:25.166887 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:25.166940 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:25.168223 (+ 1283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:25.168822 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:25.168830 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:25.169335 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:25.170003 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:25.170023 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:25.170795 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:25.170798 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:25.171719 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:25.171729 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:25.172213 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:25.172218 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:25.172383 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:25.172949 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:25.172974 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:25.174397 (+ 1423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:25.176149 (+ 1752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:25.176156 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:25.176170 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:25.176424 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:25.176707 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:25.176711 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:25.176713 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:25.176892 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:25.177362 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:25.177369 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:25.177622 (+ 253us) client_negotiation.cc:769] Sending connection context
0808 08:25:25.177790 (+ 168us) client_negotiation.cc:240] Negotiation successful
0808 08:25:25.178119 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":118,"threads_started":1}
I20260808 08:25:25.180281 1696 heartbeater.cc:347] Connected to a master server at 127.22.235.254:46527
I20260808 08:25:25.180583 1696 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:25.181188 1696 heartbeater.cc:511] Master 127.22.235.254:46527 requested a full tablet report, sending...
I20260808 08:25:25.182421 1158 ts_manager.cc:194] Registered new tserver with Master: 6b3a62b7e6a249d197410707a5bc428a (127.22.235.194:39211)
I20260808 08:25:25.182994 1158 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.235.194@KRBTEST.COM'} at 127.22.235.194:45491
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.235.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.195.keytab.
Entry for principal oryx/127.22.235.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.195.keytab.
I20260808 08:25:25.192826 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.195:0
--local_ip_for_outbound_sockets=127.22.235.195
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:46527
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.195.keytab
--principal=oryx/127.22.235.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:36355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:25:25.309165 1704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:25.309422 1704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:25.309485 1704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:25.313437 1704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:25.313500 1704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:25.313578 1704 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.195
I20260808 08:25:25.318987 1704 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:36355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.195.keytab
--principal=oryx/127.22.235.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:46527
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1704
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.195
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:25.320132 1704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:25.321238 1704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:25.330917 1717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:25.330924 1715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:25.331115 1704 server_base.cc:1034] running on GCE node
W20260808 08:25:25.330914 1714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:25.331589 1704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:25.332214 1704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:25.333365 1704 hybrid_clock.cc:648] HybridClock initialized: now 1786177525333333 us; error 44 us; skew 500 ppm
Aug 08 08:25:25 dist-test-slave-f7th krb5kdc[1076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177525, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.235.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:25.335901 1704 init.cc:377] Logged in from keytab as oryx/127.22.235.195@KRBTEST.COM (short username oryx)
I20260808 08:25:25.336751 1704 webserver.cc:460] Webserver started at http://127.22.235.195:34531/ using document root <none> and password file <none>
I20260808 08:25:25.337277 1704 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:25.337338 1704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:25.337576 1704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:25.339747 1704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-2/data/instance:
uuid: "27bb37319a6742ccb01e572b068b4bc9"
format_stamp: "Formatted at 2026-08-08 08:25:25 on dist-test-slave-f7th"
server_key: "32a9527ec83350fb4f36147745b40f02"
server_key_iv: "a37af5a1022b8dd6b463555289399548"
server_key_version: "encryptionkey@0"
I20260808 08:25:25.340263 1704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance:
uuid: "27bb37319a6742ccb01e572b068b4bc9"
format_stamp: "Formatted at 2026-08-08 08:25:25 on dist-test-slave-f7th"
server_key: "32a9527ec83350fb4f36147745b40f02"
server_key_iv: "a37af5a1022b8dd6b463555289399548"
server_key_version: "encryptionkey@0"
I20260808 08:25:25.343775 1704 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 08:25:25.346113 1730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:25.347282 1704 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 08:25:25.347381 1704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-2/wal
uuid: "27bb37319a6742ccb01e572b068b4bc9"
format_stamp: "Formatted at 2026-08-08 08:25:25 on dist-test-slave-f7th"
server_key: "32a9527ec83350fb4f36147745b40f02"
server_key_iv: "a37af5a1022b8dd6b463555289399548"
server_key_version: "encryptionkey@0"
I20260808 08:25:25.347472 1704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:25.374915 1704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:25.377722 1704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:25.377895 1704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:25.378499 1704 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:25:25.379333 1704 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:25:25.379380 1704 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:25.379426 1704 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:25:25.379453 1704 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:25.387660 1704 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.195:40217
I20260808 08:25:25.387689 1913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.195:40217 every 8 connection(s)
I20260808 08:25:25.388532 1704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
I20260808 08:25:25.388959 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 1704
I20260808 08:25:25.389091 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance
I20260808 08:25:25.389386 23471 external_mini_cluster.cc:1428] Setting key 18837854e2197ad1651c3e5d6f9e2528
Aug 08 08:25:25 dist-test-slave-f7th krb5kdc[1076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177525, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.235.195@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM
I20260808 08:25:25.401645 1213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:25.390420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:60207 (local address 127.22.235.254:46527)
0808 08:25:25.390554 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:25.390559 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:25.391578 (+ 1019us) server_negotiation.cc:407] Connection header received
0808 08:25:25.392997 (+ 1419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:25.393001 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:25.393053 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:25.393125 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:25.394250 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:25.394769 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:25.395870 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:25.396087 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:25.396174 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:25.398532 (+ 2358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:25.398536 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:25.398538 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:25.398562 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:25.399708 (+ 1146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:25.400177 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:25.400181 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:25.400183 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:25.400262 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:25.400569 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:25.400572 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:25.400574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:25.400786 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:25.400874 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:25.401351 (+ 477us) server_negotiation.cc:299] Negotiation successful
0808 08:25:25.401486 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 08:25:25.402540 1921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:25.390950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46527 (local address 127.22.235.195:60207)
0808 08:25:25.391445 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:25.391502 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:25.392803 (+ 1301us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:25.393300 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:25.393307 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:25.393647 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:25.394116 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:25.394130 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:25.394886 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:25.394889 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:25.395752 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:25.395761 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:25.396219 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:25.396223 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:25.396382 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:25.396925 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:25.396941 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:25.398370 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:25.399813 (+ 1443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:25.399819 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:25.399833 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:25.400084 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:25.400345 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:25.400348 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:25.400350 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:25.400480 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:25.400884 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:25.400891 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:25.401128 (+ 237us) client_negotiation.cc:769] Sending connection context
0808 08:25:25.401292 (+ 164us) client_negotiation.cc:240] Negotiation successful
0808 08:25:25.401596 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":146,"threads_started":1}
I20260808 08:25:25.403657 1916 heartbeater.cc:347] Connected to a master server at 127.22.235.254:46527
I20260808 08:25:25.403910 1916 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:25.404382 1916 heartbeater.cc:511] Master 127.22.235.254:46527 requested a full tablet report, sending...
I20260808 08:25:25.405422 1157 ts_manager.cc:194] Registered new tserver with Master: 27bb37319a6742ccb01e572b068b4bc9 (127.22.235.195:40217)
I20260808 08:25:25.405970 1157 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.235.195@KRBTEST.COM'} at 127.22.235.195:60207
I20260808 08:25:25.415601 23471 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.22.235.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.235.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.235.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.253.keytab.
Entry for principal oryx/127.22.235.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.253.keytab.
I20260808 08:25:25.441083 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.253:42359
--webserver_interface=127.22.235.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.253.keytab
--principal=oryx/127.22.235.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:36355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:46527,127.22.235.253:42359
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:25:25.552289 1925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:25.552546 1925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:25.552601 1925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:25.556486 1925 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:25:25.556537 1925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:25.556555 1925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:25.556577 1925 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:25:25.556591 1925 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:25:25.561697 1925 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:36355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:46527,127.22.235.253:42359
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.253.keytab
--principal=oryx/127.22.235.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.253:42359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.235.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1925
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
W20260808 08:25:25.561874 1925 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.22.235.254:46527,127.22.235.253:42359'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260808 08:25:25.562668 1925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:25.563591 1925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 08:25:25.571686 1925 server_base.cc:1034] running on GCE node
W20260808 08:25:25.571698 1933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:25.571692 1932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:25.571938 1937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:25.572448 1925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:25.573534 1925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:25.574683 1925 hybrid_clock.cc:648] HybridClock initialized: now 1786177525574659 us; error 35 us; skew 500 ppm
Aug 08 08:25:25 dist-test-slave-f7th krb5kdc[1076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177525, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.235.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:25.577358 1925 init.cc:377] Logged in from keytab as oryx/127.22.235.253@KRBTEST.COM (short username oryx)
I20260808 08:25:25.578200 1925 webserver.cc:460] Webserver started at http://127.22.235.253:33003/ using document root <none> and password file <none>
I20260808 08:25:25.578732 1925 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:25.578794 1925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:25.578999 1925 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:25.581037 1925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-1/data/instance:
uuid: "ad643f833e2a4adc89d9c6c143dbb971"
format_stamp: "Formatted at 2026-08-08 08:25:25 on dist-test-slave-f7th"
server_key: "961b44462db1b79bbc8c9d0d1db16ae3"
server_key_iv: "7a77f4d420eb2f134f3006ba2b4dcd41"
server_key_version: "encryptionkey@0"
I20260808 08:25:25.581573 1925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-1/wal/instance:
uuid: "ad643f833e2a4adc89d9c6c143dbb971"
format_stamp: "Formatted at 2026-08-08 08:25:25 on dist-test-slave-f7th"
server_key: "961b44462db1b79bbc8c9d0d1db16ae3"
server_key_iv: "7a77f4d420eb2f134f3006ba2b4dcd41"
server_key_version: "encryptionkey@0"
I20260808 08:25:25.585098 1925 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 08:25:25.587453 1950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:25.588717 1925 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:25:25.588840 1925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-1/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-1/wal
uuid: "ad643f833e2a4adc89d9c6c143dbb971"
format_stamp: "Formatted at 2026-08-08 08:25:25 on dist-test-slave-f7th"
server_key: "961b44462db1b79bbc8c9d0d1db16ae3"
server_key_iv: "7a77f4d420eb2f134f3006ba2b4dcd41"
server_key_version: "encryptionkey@0"
I20260808 08:25:25.588939 1925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:25.611580 1925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:25.614315 1925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:25.614490 1925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:25.622007 1925 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.253:42359
I20260808 08:25:25.622256 2041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.253:42359 every 8 connection(s)
I20260808 08:25:25.623018 1925 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-1/data/info.pb
I20260808 08:25:25.625768 2046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:25.626895 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 1925
I20260808 08:25:25.630090 2046 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 08:25:25 dist-test-slave-f7th krb5kdc[1076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177524, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.235.253@KRBTEST.COM
Aug 08 08:25:25 dist-test-slave-f7th krb5kdc[1076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177525, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.235.253@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM
I20260808 08:25:25.641073 1213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:25.631944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44718 (local address 127.22.235.254:46527)
0808 08:25:25.632087 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:25.632092 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:25.632383 (+ 291us) server_negotiation.cc:407] Connection header received
0808 08:25:25.632526 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:25.632529 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:25.632579 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:25.632694 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:25.633493 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:25.634169 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:25.635275 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:25.635483 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:25.635557 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:25.637906 (+ 2349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:25.637909 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:25.637911 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:25.637935 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:25.639216 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:25.639700 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:25.639703 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:25.639705 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:25.639792 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:25.640038 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:25.640040 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:25.640041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:25.640235 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:25.640310 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:25.640848 (+ 538us) server_negotiation.cc:299] Negotiation successful
0808 08:25:25.640950 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 08:25:25.641884 2050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:25.628555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51788 (local address 127.22.235.253:42359)
0808 08:25:25.629004 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:25.629023 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:25.629066 (+ 43us) server_negotiation.cc:407] Connection header received
0808 08:25:25.629932 (+ 866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:25.629956 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:25.630369 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:25.630741 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:25.631401 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:25.632640 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:25.633726 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:25.634391 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:25.634505 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:25.637248 (+ 2743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:25.637256 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:25.637268 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:25.637295 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:25.638956 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:25.639393 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:25.639398 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:25.639400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:25.639517 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:25.639784 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:25.639786 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:25.639787 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:25.640245 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:25.640450 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:25.640608 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 08:25:25.640912 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":294,"thread_start_us":111,"threads_started":1}
I20260808 08:25:25.641884 2051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:25.631860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46527 (local address 127.0.0.1:44718)
0808 08:25:25.632261 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:25.632309 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:25.632425 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:25.632741 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:25.632747 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:25.633216 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:25.633367 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:25.633375 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:25.634294 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:25.634297 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:25.635146 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:25.635159 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:25.635628 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:25.635632 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:25.635752 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:25.636220 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:25.636240 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:25.637764 (+ 1524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:25.639322 (+ 1558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:25.639329 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:25.639337 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:25.639614 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:25.639872 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:25.639875 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:25.639877 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:25.639957 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:25.640329 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:25.640335 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:25.640489 (+ 154us) client_negotiation.cc:769] Sending connection context
0808 08:25:25.640677 (+ 188us) client_negotiation.cc:240] Negotiation successful
0808 08:25:25.640976 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"mutex_wait_us":95,"thread_start_us":156,"threads_started":1}
I20260808 08:25:25.643913 2046 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.235.253" port: 42359 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 08:25:25 dist-test-slave-f7th krb5kdc[1076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177525, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.235.253@KRBTEST.COM for oryx/127.22.235.253@KRBTEST.COM
I20260808 08:25:25.650677 2051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:25.644331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.253:42359 (local address 127.0.0.1:51792)
0808 08:25:25.644414 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:25.644427 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:25.644492 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:25.644697 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:25.644699 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:25.644814 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:25.644897 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:25.644901 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:25.645852 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:25.645854 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:25.646336 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:25.646344 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:25.646696 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:25.646698 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:25.646764 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:25.647175 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:25.647188 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:25.648237 (+ 1049us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:25.649540 (+ 1303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:25.649542 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:25.649544 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:25.649713 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:25.649901 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:25.649903 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:25.649904 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:25.649997 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:25.650298 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:25.650301 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:25.650374 (+ 73us) client_negotiation.cc:769] Sending connection context
0808 08:25:25.650451 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 08:25:25.650550 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260808 08:25:25.650715 2050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:25.644397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51792 (local address 127.22.235.253:42359)
0808 08:25:25.644488 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:25.644491 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:25.644505 (+ 14us) server_negotiation.cc:407] Connection header received
0808 08:25:25.644562 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:25.644564 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:25.644597 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:25.644677 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:25.644974 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:25.645757 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:25.646419 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:25.646624 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:25.646689 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:25.648337 (+ 1648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:25.648340 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:25.648342 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:25.648360 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:25.649445 (+ 1085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:25.649780 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:25.649782 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:25.649783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:25.649844 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:25.650062 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:25.650064 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:25.650065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:25.650220 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:25.650298 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:25.650493 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 08:25:25.650602 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":35}
I20260808 08:25:25.653466 2046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971: Bootstrap starting.
I20260808 08:25:25.655635 2046 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:25.656322 2046 log.cc:824] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971: Log is configured to *not* fsync() on all Append() calls
I20260808 08:25:25.658120 2046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971: No bootstrap required, opened a new log
I20260808 08:25:25.660964 2046 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 42359 } }
I20260808 08:25:25.661190 2046 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:25.661259 2046 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad643f833e2a4adc89d9c6c143dbb971, State: Initialized, Role: FOLLOWER
I20260808 08:25:25.661762 2046 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 42359 } }
I20260808 08:25:25.662202 2054 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 42359 } } }
I20260808 08:25:25.662312 2054 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 08:25:25.662904 2046 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 08:25:25.667327 2071 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 08:25:25.667405 2071 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 08:25:25.668856 1925 master_runner.cc:428] Detected that this master ad643f833e2a4adc89d9c6c143dbb971 is joining an existing cluster
I20260808 08:25:25.668906 1925 master_runner.cc:432] Initiating AddMaster RPC to add 127.22.235.253:42359
I20260808 08:25:25.678354 1213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:25.670803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44728 (local address 127.22.235.254:46527)
0808 08:25:25.670919 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:25.670923 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:25.671078 (+ 155us) server_negotiation.cc:407] Connection header received
0808 08:25:25.671239 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:25.671243 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:25.671294 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:25.671347 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:25.671915 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:25.672671 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:25.673611 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:25.673843 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:25.673948 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:25.675222 (+ 1274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:25.675226 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:25.675229 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:25.675255 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:25.676761 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:25.677192 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:25.677196 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:25.677197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:25.677298 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:25.677571 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:25.677574 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:25.677576 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:25.677753 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:25.677840 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:25.678099 (+ 259us) server_negotiation.cc:299] Negotiation successful
0808 08:25:25.678207 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 08:25:25.678367 2087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:25.670736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46527 (local address 127.0.0.1:44728)
0808 08:25:25.671012 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:25.671026 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:25.671130 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:25.671474 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:25.671476 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:25.671692 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:25.671824 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:25.671831 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:25.672790 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:25.672793 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:25.673502 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:25.673512 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:25.673954 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:25.673958 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:25.674131 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:25.674656 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:25.674672 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:25.675112 (+ 440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:25.676867 (+ 1755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:25.676870 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:25.676873 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:25.677090 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:25.677395 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:25.677397 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:25.677398 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:25.677473 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:25.677841 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:25.677844 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:25.677944 (+ 100us) client_negotiation.cc:769] Sending connection context
0808 08:25:25.678034 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 08:25:25.678164 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":62,"thread_start_us":107,"threads_started":1}
I20260808 08:25:25.679615 2050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:25.671229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51804 (local address 127.22.235.253:42359)
0808 08:25:25.671319 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:25.671323 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:25.671587 (+ 264us) server_negotiation.cc:407] Connection header received
0808 08:25:25.671674 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:25.671677 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:25.671718 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:25.671795 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:25.672288 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:25.672966 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:25.674228 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:25.674547 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:25.674667 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:25.676208 (+ 1541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:25.676212 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:25.676214 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:25.676240 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:25.678075 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:25.678516 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:25.678520 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:25.678521 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:25.678593 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:25.678834 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:25.678837 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:25.678839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:25.679027 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:25.679091 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:25.679366 (+ 275us) server_negotiation.cc:299] Negotiation successful
0808 08:25:25.679466 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":36}
I20260808 08:25:25.680027 2088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:25.671183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.253:42359 (local address 127.0.0.1:51804)
0808 08:25:25.671491 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:25.671505 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:25.671605 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:25.671826 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:25.671829 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:25.672046 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:25.672175 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:25.672182 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:25.673090 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:25.673094 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:25.674116 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:25.674125 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:25.674667 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:25.674671 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:25.674778 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:25.675480 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:25.675499 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:25.675856 (+ 357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:25.678184 (+ 2328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:25.678187 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:25.678189 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:25.678425 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:25.678678 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:25.678680 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:25.678682 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:25.678754 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:25.679146 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:25.679148 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:25.679225 (+ 77us) client_negotiation.cc:769] Sending connection context
0808 08:25:25.679736 (+ 511us) client_negotiation.cc:240] Negotiation successful
0808 08:25:25.679854 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"mutex_wait_us":45,"thread_start_us":80,"threads_started":1}
Aug 08 08:25:25 dist-test-slave-f7th krb5kdc[1076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177524, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.235.254@KRBTEST.COM for oryx/127.22.235.253@KRBTEST.COM
I20260808 08:25:25.690881 2089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:25.682792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.253:42359 (local address 127.0.0.1:51814)
0808 08:25:25.683128 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:25.683177 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:25.683303 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:25.683577 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:25.683581 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:25.683886 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:25.683981 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:25.683986 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:25.684914 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:25.684917 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:25.685520 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:25.685530 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:25.686036 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:25.686038 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:25.686110 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:25.686530 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:25.686541 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:25.687824 (+ 1283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:25.689522 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:25.689532 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:25.689538 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:25.689754 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:25.690011 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:25.690013 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:25.690014 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:25.690081 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:25.690422 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:25.690426 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:25.690524 (+ 98us) client_negotiation.cc:769] Sending connection context
0808 08:25:25.690631 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 08:25:25.690764 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":89,"threads_started":1}
I20260808 08:25:25.690940 2050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:25.682796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51814 (local address 127.22.235.253:42359)
0808 08:25:25.682909 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:25.682912 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:25.683248 (+ 336us) server_negotiation.cc:407] Connection header received
0808 08:25:25.683422 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:25.683425 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:25.683474 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:25.683552 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:25.684082 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:25.684827 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:25.685617 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:25.685940 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:25.686040 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:25.687946 (+ 1906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:25.687950 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:25.687953 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:25.687986 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:25.689425 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:25.689835 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:25.689838 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:25.689840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:25.689920 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:25.690150 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:25.690153 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:25.690154 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:25.690340 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:25.690433 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:25.690696 (+ 263us) server_negotiation.cc:299] Negotiation successful
0808 08:25:25.690798 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 08:25:25.692198 1157 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.22.235.253:42359: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971"
member_type: NON_VOTER
last_known_addr {
host: "127.22.235.253"
port: 42359
}
attrs {
promote: true
}
}
dest_uuid: "00ae12b3a32549bb9d8a3092fa6e0139"
cas_config_opid_index: -1
I20260808 08:25:25.692816 1157 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: NON_VOTER last_known_addr { host: "127.22.235.253" port: 42359 } attrs { promote: true } }
I20260808 08:25:25.695147 1997 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:25.696470 1997 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [term 1 FOLLOWER]: Refusing update from remote peer 00ae12b3a32549bb9d8a3092fa6e0139: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260808 08:25:25.697278 2090 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER ad643f833e2a4adc89d9c6c143dbb971 (127.22.235.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: NON_VOTER last_known_addr { host: "127.22.235.253" port: 42359 } attrs { promote: true } } }
I20260808 08:25:25.698429 2091 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: NON_VOTER last_known_addr { host: "127.22.235.253" port: 42359 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:25:25.699080 2090 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.22.235.253:42359
I20260808 08:25:25.699560 1925 master.cc:537] Master@127.22.235.253:42359 shutting down...
I20260808 08:25:25.699738 2090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: NON_VOTER last_known_addr { host: "127.22.235.253" port: 42359 } attrs { promote: true } } }
I20260808 08:25:25.699823 2090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 [sys.catalog]: This master's current role is: LEADER
W20260808 08:25:25.700259 1126 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260808 08:25:25.703145 1126 consensus_peers.cc:487] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 -> Peer ad643f833e2a4adc89d9c6c143dbb971 (127.22.235.253:42359): Couldn't send request to peer ad643f833e2a4adc89d9c6c143dbb971. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 08:25:25.704013 1925 raft_consensus.cc:2229] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 08:25:25.704260 1925 raft_consensus.cc:2258] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:25:25.704360 1925 tablet_replica.cc:330] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971: stopping tablet replica
I20260808 08:25:25.709522 1925 master.cc:552] Master@127.22.235.253:42359 shutdown complete.
I20260808 08:25:25.709570 1925 master_runner.cc:305] Clearing existing system tablet
I20260808 08:25:25.709774 1925 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:25.709823 1925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:25.710834 1925 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 08:25:25.711567 2111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:25.711760 1925 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:25.711882 1925 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 08:25:25.711938 1925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-1/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-1/wal
uuid: "ad643f833e2a4adc89d9c6c143dbb971"
format_stamp: "Formatted at 2026-08-08 08:25:25 on dist-test-slave-f7th"
server_key: "961b44462db1b79bbc8c9d0d1db16ae3"
server_key_iv: "7a77f4d420eb2f134f3006ba2b4dcd41"
server_key_version: "encryptionkey@0"
WARNING: no policy specified for oryx/127.22.235.252@KRBTEST.COM; defaulting to no policy
I20260808 08:25:25.713698 1925 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971: Deleting tablet data with delete state TABLET_DATA_DELETED
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.235.252@KRBTEST.COM" created.
I20260808 08:25:25.716184 1925 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 08:25:25.716305 1925 log.cc:1196] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260808 08:25:25.716631 1925 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971: Deleting consensus metadata
I20260808 08:25:25.716825 1925 master_runner.cc:315] Copying system tablet from 127.22.235.254:46527
I20260808 08:25:25.717995 1925 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971: tablet copy: Beginning tablet copy session from remote peer at address 127.22.235.254:46527
I20260808 08:25:25.726013 1213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:25.718546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44730 (local address 127.22.235.254:46527)
0808 08:25:25.718665 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:25.718670 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:25.718894 (+ 224us) server_negotiation.cc:407] Connection header received
0808 08:25:25.719040 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:25.719044 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:25.719087 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:25.719151 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:25.719581 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:25.720166 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:25.721033 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:25.721247 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:25.721332 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:25.722921 (+ 1589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:25.722926 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:25.722928 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:25.722956 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:25.724339 (+ 1383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:25.724801 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:25.724805 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:25.724807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:25.724880 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:25.725132 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:25.725135 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:25.725136 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:25.725300 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:25.725382 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:25.725761 (+ 379us) server_negotiation.cc:299] Negotiation successful
0808 08:25:25.725862 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 08:25:25.726034 2121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:25.718471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46527 (local address 127.0.0.1:44730)
0808 08:25:25.718815 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:25.718830 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:25.718936 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:25.719180 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:25.719183 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:25.719361 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:25.719453 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:25.719459 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:25.720268 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:25.720271 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:25.720911 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:25.720921 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:25.721343 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:25.721347 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:25.721439 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:25.722304 (+ 865us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:25.722317 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:25.722759 (+ 442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:25.724441 (+ 1682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:25.724444 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:25.724446 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:25.724722 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:25.724961 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:25.724964 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:25.724967 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:25.725053 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:25.725514 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:25.725517 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:25.725603 (+ 86us) client_negotiation.cc:769] Sending connection context
0808 08:25:25.725688 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 08:25:25.725824 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":95,"threads_started":1}
I20260808 08:25:25.726804 1190 tablet_copy_service.cc:140] P 00ae12b3a32549bb9d8a3092fa6e0139: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer ad643f833e2a4adc89d9c6c143dbb971 ({username='oryx', principal='oryx/127.22.235.253@KRBTEST.COM'} at 127.0.0.1:44730)
I20260808 08:25:25.726962 1190 tablet_copy_service.cc:161] P 00ae12b3a32549bb9d8a3092fa6e0139: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer ad643f833e2a4adc89d9c6c143dbb971 at {username='oryx', principal='oryx/127.22.235.253@KRBTEST.COM'} at 127.0.0.1:44730: session id = ad643f833e2a4adc89d9c6c143dbb971-00000000000000000000000000000000
I20260808 08:25:25.728571 1190 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 08:25:25.730103 1925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.235.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.252.keytab.
Entry for principal oryx/127.22.235.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.252.keytab.
I20260808 08:25:25.731523 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.252:42345
--webserver_interface=127.22.235.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.252.keytab
--principal=oryx/127.22.235.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:36355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:46527,127.22.235.253:42359,127.22.235.252:42345
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 08:25:25.732225 1925 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971: tablet copy: Starting download of 0 data blocks...
I20260808 08:25:25.732407 1925 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971: tablet copy: Starting download of 1 WAL segments...
I20260808 08:25:25.734784 1925 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 08:25:25.736188 1190 tablet_copy_service.cc:342] P 00ae12b3a32549bb9d8a3092fa6e0139: Request end of tablet copy session ad643f833e2a4adc89d9c6c143dbb971-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.22.235.253@KRBTEST.COM'} at 127.0.0.1:44730
I20260808 08:25:25.736281 1190 tablet_copy_service.cc:433] P 00ae12b3a32549bb9d8a3092fa6e0139: ending tablet copy session ad643f833e2a4adc89d9c6c143dbb971-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer ad643f833e2a4adc89d9c6c143dbb971
I20260808 08:25:25.742453 1925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:25.744025 2130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:25.744052 2124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:25.744385 1925 server_base.cc:1034] running on GCE node
W20260808 08:25:25.748744 2125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:25.748945 1925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:25.749218 1925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:25.750355 1925 hybrid_clock.cc:648] HybridClock initialized: now 1786177525750340 us; error 31 us; skew 500 ppm
Aug 08 08:25:25 dist-test-slave-f7th krb5kdc[1076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177525, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.235.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:25.751820 1925 init.cc:377] Logged in from keytab as oryx/127.22.235.253@KRBTEST.COM (short username oryx)
I20260808 08:25:25.752223 1925 webserver.cc:460] Webserver started at http://127.22.235.253:44351/ using document root <none> and password file <none>
I20260808 08:25:25.752395 1925 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:25.752430 1925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:25.753562 1925 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 08:25:25.754354 2139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:25.754556 1925 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 08:25:25.754629 1925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-1/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-1/wal
uuid: "ad643f833e2a4adc89d9c6c143dbb971"
format_stamp: "Formatted at 2026-08-08 08:25:25 on dist-test-slave-f7th"
server_key: "961b44462db1b79bbc8c9d0d1db16ae3"
server_key_iv: "7a77f4d420eb2f134f3006ba2b4dcd41"
server_key_version: "encryptionkey@0"
I20260808 08:25:25.754705 1925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:25.767114 1925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:25.769089 1925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:25.773603 1925 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.253:42359
I20260808 08:25:25.774174 1925 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-1/data/info.pb
I20260808 08:25:25.775763 2239 sys_catalog.cc:263] Verifying existing consensus state
I20260808 08:25:25.776559 2239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971: Bootstrap starting.
I20260808 08:25:25.780807 2238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.253:42359 every 8 connection(s)
I20260808 08:25:25.784454 2239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 08:25:25.784766 2239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971: Bootstrap complete.
I20260808 08:25:25.785218 2239 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: NON_VOTER last_known_addr { host: "127.22.235.253" port: 42359 } attrs { promote: true } }
I20260808 08:25:25.785322 2239 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ad643f833e2a4adc89d9c6c143dbb971, State: Initialized, Role: LEARNER
I20260808 08:25:25.785450 2239 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: NON_VOTER last_known_addr { host: "127.22.235.253" port: 42359 } attrs { promote: true } }
I20260808 08:25:25.785758 2244 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: NON_VOTER last_known_addr { host: "127.22.235.253" port: 42359 } attrs { promote: true } } }
I20260808 08:25:25.785912 2244 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [sys.catalog]: This master's current role is: LEARNER
I20260808 08:25:25.785897 2239 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:25:25.788676 2261 catalog_manager.cc:1259] Loaded cluster ID: aafd38e6ed664f44b830a3f06aac46a6
I20260808 08:25:25.788769 2261 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971: loading cluster ID for follower catalog manager: success
I20260808 08:25:25.790349 2261 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971: acquiring CA information for follower catalog manager: success
I20260808 08:25:25.791404 2261 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 08:25:25.882790 2122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:25.883047 2122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:25.883085 2122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:25.887487 2122 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:25:25.887543 2122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:25.887562 2122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:25.887576 2122 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:25:25.887590 2122 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:25:25.892594 2122 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:36355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:46527,127.22.235.253:42359,127.22.235.252:42345
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.252.keytab
--principal=oryx/127.22.235.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.252:42345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.235.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:25.893559 2122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:25.894503 2122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:25.902585 2271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:25.902647 2268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:25.902772 2122 server_base.cc:1034] running on GCE node
W20260808 08:25:25.902593 2269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:25.903430 2122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:25.904477 2122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:25.905642 2122 hybrid_clock.cc:648] HybridClock initialized: now 1786177525905620 us; error 36 us; skew 500 ppm
Aug 08 08:25:25 dist-test-slave-f7th krb5kdc[1076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177525, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.235.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:25.908315 2122 init.cc:377] Logged in from keytab as oryx/127.22.235.252@KRBTEST.COM (short username oryx)
I20260808 08:25:25.909179 2122 webserver.cc:460] Webserver started at http://127.22.235.252:36529/ using document root <none> and password file <none>
I20260808 08:25:25.909688 2122 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:25.909744 2122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:25.909914 2122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:25.911875 2122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-2/data/instance:
uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558"
format_stamp: "Formatted at 2026-08-08 08:25:25 on dist-test-slave-f7th"
server_key: "b408d220b8d4132de2eb13587d4636d9"
server_key_iv: "1ff6fe6f978d2efd4459c6356a4a30a3"
server_key_version: "encryptionkey@0"
I20260808 08:25:25.912359 2122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-2/wal/instance:
uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558"
format_stamp: "Formatted at 2026-08-08 08:25:25 on dist-test-slave-f7th"
server_key: "b408d220b8d4132de2eb13587d4636d9"
server_key_iv: "1ff6fe6f978d2efd4459c6356a4a30a3"
server_key_version: "encryptionkey@0"
I20260808 08:25:25.915954 2122 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 08:25:25.918402 2280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:25.919595 2122 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:25:25.919699 2122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-2/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-2/wal
uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558"
format_stamp: "Formatted at 2026-08-08 08:25:25 on dist-test-slave-f7th"
server_key: "b408d220b8d4132de2eb13587d4636d9"
server_key_iv: "1ff6fe6f978d2efd4459c6356a4a30a3"
server_key_version: "encryptionkey@0"
I20260808 08:25:25.919790 2122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:25.929848 2122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:25.932711 2122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:25.932871 2122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:25.939476 2122 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.252:42345
I20260808 08:25:25.939747 2365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.252:42345 every 8 connection(s)
I20260808 08:25:25.940529 2122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-2/data/info.pb
I20260808 08:25:25.943360 2366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:25.947201 2366 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } has no permanent_uuid. Determining permanent_uuid...
I20260808 08:25:25.948854 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 2122
Aug 08 08:25:25 dist-test-slave-f7th krb5kdc[1076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177525, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.235.252@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM
Aug 08 08:25:25 dist-test-slave-f7th krb5kdc[1076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177524, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.235.252@KRBTEST.COM
I20260808 08:25:25.960774 1213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:25.949083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44742 (local address 127.22.235.254:46527)
0808 08:25:25.949215 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:25.949219 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:25.950110 (+ 891us) server_negotiation.cc:407] Connection header received
0808 08:25:25.951092 (+ 982us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:25.951096 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:25.951156 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:25.951232 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:25.952271 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:25.953138 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:25.954522 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:25.954866 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:25.954952 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:25.957416 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:25.957419 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:25.957422 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:25.957448 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:25.958811 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:25.959316 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:25.959319 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:25.959321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:25.959398 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:25.959699 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:25.959702 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:25.959704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:25.959894 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:25.960026 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:25.960469 (+ 443us) server_negotiation.cc:299] Negotiation successful
0808 08:25:25.960592 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 08:25:25.961557 2375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:25.949754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46346 (local address 127.22.235.252:42345)
0808 08:25:25.950158 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:25.950169 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:25.950209 (+ 40us) server_negotiation.cc:407] Connection header received
0808 08:25:25.950871 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:25.950891 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:25.951126 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:25.951290 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:25.952128 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:25.953396 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:25.954244 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:25.954681 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:25.954940 (+ 259us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:25.957722 (+ 2782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:25.957729 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:25.957741 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:25.957762 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:25.959202 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:25.959592 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:25.959598 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:25.959600 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:25.959675 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:25.959968 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:25.959971 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:25.959973 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:25.960249 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:25.960441 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:25.960679 (+ 238us) server_negotiation.cc:299] Negotiation successful
0808 08:25:25.960811 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":131,"threads_started":1}
I20260808 08:25:25.961565 2372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:25.949457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46527 (local address 127.0.0.1:44742)
0808 08:25:25.949944 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:25.950019 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:25.950875 (+ 856us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:25.951300 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:25.951305 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:25.951742 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:25.952115 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:25.952130 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:25.953276 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:25.953279 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:25.954374 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:25.954386 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:25.955132 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:25.955135 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:25.955221 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:25.955688 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:25.955705 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:25.957284 (+ 1579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:25.958907 (+ 1623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:25.958915 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:25.958925 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:25.959227 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:25.959478 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:25.959481 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:25.959483 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:25.959613 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:25.960013 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:25.960019 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:25.960258 (+ 239us) client_negotiation.cc:769] Sending connection context
0808 08:25:25.960404 (+ 146us) client_negotiation.cc:240] Negotiation successful
0808 08:25:25.960717 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":120,"threads_started":1}
I20260808 08:25:25.963325 2366 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.235.253" port: 42359 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 08:25:25 dist-test-slave-f7th krb5kdc[1076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177525, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.235.252@KRBTEST.COM for oryx/127.22.235.253@KRBTEST.COM
I20260808 08:25:25.970655 2372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:25.963808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.253:42359 (local address 127.0.0.1:51828)
0808 08:25:25.963914 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:25.963926 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:25.963986 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:25.964353 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:25.964356 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:25.964517 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:25.964605 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:25.964634 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:25.965484 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:25.965488 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:25.966005 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:25.966012 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:25.966433 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:25.966436 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:25.966536 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:25.966885 (+ 349us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:25.966895 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:25.967944 (+ 1049us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:25.969350 (+ 1406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:25.969354 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:25.969356 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:25.969543 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:25.969789 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:25.969792 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:25.969794 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:25.969869 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:25.970212 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:25.970215 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:25.970299 (+ 84us) client_negotiation.cc:769] Sending connection context
0808 08:25:25.970382 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 08:25:25.970512 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 08:25:25.970727 2376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:25.963923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51828 (local address 127.22.235.253:42359)
0808 08:25:25.964162 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:25.964166 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:25.964176 (+ 10us) server_negotiation.cc:407] Connection header received
0808 08:25:25.964211 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:25.964213 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:25.964250 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:25.964324 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:25.964732 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:25.965377 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:25.966111 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:25.966338 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:25.966425 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:25.968071 (+ 1646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:25.968074 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:25.968076 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:25.968098 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:25.969250 (+ 1152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:25.969632 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:25.969634 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:25.969636 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:25.969708 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:25.969946 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:25.969950 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:25.969951 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:25.970123 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:25.970208 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:25.970444 (+ 236us) server_negotiation.cc:299] Negotiation successful
0808 08:25:25.970552 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":82,"threads_started":1}
I20260808 08:25:25.971393 2366 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.235.252" port: 42345 } has no permanent_uuid. Determining permanent_uuid...
I20260808 08:25:25.973960 1479 heartbeater.cc:503] Master 127.22.235.254:46527 was elected leader, sending a full tablet report...
Aug 08 08:25:25 dist-test-slave-f7th krb5kdc[1076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177525, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.235.252@KRBTEST.COM for oryx/127.22.235.252@KRBTEST.COM
I20260808 08:25:25.978027 2372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:25.971788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:42345 (local address 127.0.0.1:46356)
0808 08:25:25.971874 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:25.971887 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:25.971949 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:25.972120 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:25.972121 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:25.972237 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:25.972299 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:25.972303 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:25.972983 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:25.972986 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:25.973448 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:25.973455 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:25.973812 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:25.973815 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:25.973885 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:25.974293 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:25.974307 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:25.975581 (+ 1274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:25.976991 (+ 1410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:25.976994 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:25.976995 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:25.977159 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:25.977359 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:25.977361 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:25.977362 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:25.977422 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:25.977683 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:25.977685 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:25.977751 (+ 66us) client_negotiation.cc:769] Sending connection context
0808 08:25:25.977818 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 08:25:25.977909 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260808 08:25:25.978091 2375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:25.971848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46356 (local address 127.22.235.252:42345)
0808 08:25:25.971939 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:25.971942 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:25.971953 (+ 11us) server_negotiation.cc:407] Connection header received
0808 08:25:25.972012 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:25.972014 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:25.972045 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:25.972103 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:25.972371 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:25.972882 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:25.973529 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:25.973734 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:25.973797 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:25.975678 (+ 1881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:25.975681 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:25.975683 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:25.975703 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:25.976916 (+ 1213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:25.977236 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:25.977238 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:25.977239 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:25.977301 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:25.977478 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:25.977480 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:25.977481 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:25.977624 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:25.977691 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:25.977850 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 08:25:25.977959 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":33}
I20260808 08:25:25.981135 2366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558: Bootstrap starting.
I20260808 08:25:25.983384 2366 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:25.984073 2366 log.cc:824] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558: Log is configured to *not* fsync() on all Append() calls
I20260808 08:25:25.985868 2366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558: No bootstrap required, opened a new log
I20260808 08:25:25.988991 2366 raft_consensus.cc:348] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 42359 } } peers { permanent_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 42345 } }
I20260808 08:25:25.989197 2366 raft_consensus.cc:374] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:25.989236 2366 raft_consensus.cc:729] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b79f9c81ea0b4abeadd4ac1dbf54f558, State: Initialized, Role: FOLLOWER
I20260808 08:25:25.989602 2366 consensus_queue.cc:260] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 42359 } } peers { permanent_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 42345 } }
I20260808 08:25:25.990003 2380 sys_catalog.cc:455] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 42359 } } peers { permanent_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 42345 } } }
I20260808 08:25:25.990123 2380 sys_catalog.cc:458] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 08:25:25.990701 2366 sys_catalog.cc:564] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 08:25:25.995041 2395 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 08:25:25.995110 2395 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 08:25:25.998553 2122 master_runner.cc:428] Detected that this master b79f9c81ea0b4abeadd4ac1dbf54f558 is joining an existing cluster
I20260808 08:25:25.998620 2122 master_runner.cc:432] Initiating AddMaster RPC to add 127.22.235.252:42345
I20260808 08:25:26.014917 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 1251
I20260808 08:25:26.047508 2375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.001810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46366 (local address 127.22.235.252:42345)
0808 08:25:26.001919 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:26.001923 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:26.002285 (+ 362us) server_negotiation.cc:407] Connection header received
0808 08:25:26.002409 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:26.002412 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:26.002447 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:26.002490 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:26.002947 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.003659 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.005155 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.005483 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.005592 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.043642 (+ 38050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:26.043646 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:26.043648 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:26.043670 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:26.045689 (+ 2019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:26.046168 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:26.046172 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:26.046174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:26.046250 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:26.046504 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:26.046507 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:26.046508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:26.046746 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:26.046838 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:26.047187 (+ 349us) server_negotiation.cc:299] Negotiation successful
0808 08:25:26.047320 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":48}
I20260808 08:25:26.048295 2410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.001673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:42345 (local address 127.0.0.1:46366)
0808 08:25:26.002223 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:26.002235 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:26.002331 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:26.002599 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:26.002602 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:26.002756 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:26.002854 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:26.002860 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.003774 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.003777 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:26.005051 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:26.005060 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.005686 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.005689 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:26.005766 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.006129 (+ 363us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:26.006139 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:26.043502 (+ 37363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:26.045800 (+ 2298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:26.045803 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:26.045805 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:26.046064 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:26.046342 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:26.046345 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:26.046346 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:26.046421 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:26.046898 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:26.046900 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:26.047025 (+ 125us) client_negotiation.cc:769] Sending connection context
0808 08:25:26.047950 (+ 925us) client_negotiation.cc:240] Negotiation successful
0808 08:25:26.048115 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"mutex_wait_us":3,"thread_start_us":84,"threads_started":1}
I20260808 08:25:26.048899 2409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.001334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.253:42359 (local address 127.0.0.1:51838)
0808 08:25:26.001911 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:26.001924 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:26.002030 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:26.002351 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:26.002354 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:26.002617 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:26.002727 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:26.002734 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.003774 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.003777 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:26.004710 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:26.004721 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.005381 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.005385 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:26.005531 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.042546 (+ 37015us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:26.042596 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:26.043123 (+ 527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:26.047250 (+ 4127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:26.047253 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:26.047255 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:26.047479 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:26.047806 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:26.047809 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:26.047811 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:26.047903 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:26.048359 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:26.048362 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:26.048462 (+ 100us) client_negotiation.cc:769] Sending connection context
0808 08:25:26.048563 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 08:25:26.048737 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"mutex_wait_us":72,"thread_start_us":76,"threads_started":1}
I20260808 08:25:26.049681 2376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.001540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51838 (local address 127.22.235.253:42359)
0808 08:25:26.001663 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:26.001667 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:26.002012 (+ 345us) server_negotiation.cc:407] Connection header received
0808 08:25:26.002115 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:26.002118 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:26.002168 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:26.002235 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:26.002857 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.003663 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.004839 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.005175 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.005258 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.044837 (+ 39579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:26.044841 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:26.044844 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:26.044877 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:26.047139 (+ 2262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:26.047603 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:26.047606 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:26.047608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:26.047699 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:26.047999 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:26.048002 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:26.048004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:26.048220 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:26.048311 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:26.049376 (+ 1065us) server_negotiation.cc:299] Negotiation successful
0808 08:25:26.049527 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 08:25:26.050874 1213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.001034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44744 (local address 127.22.235.254:46527)
0808 08:25:26.001165 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:26.001170 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:26.001288 (+ 118us) server_negotiation.cc:407] Connection header received
0808 08:25:26.001405 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:26.001408 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:26.001460 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:26.001546 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:26.002453 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.003283 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.004498 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.004862 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.004940 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.044086 (+ 39146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:26.044090 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:26.044093 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:26.044128 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:26.046373 (+ 2245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:26.049217 (+ 2844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:26.049221 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:26.049223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:26.049322 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:26.049649 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:26.049652 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:26.049654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:26.049879 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:26.049948 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:26.050601 (+ 653us) server_negotiation.cc:299] Negotiation successful
0808 08:25:26.050731 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 08:25:26.051432 2408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.000938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46527 (local address 127.0.0.1:44744)
0808 08:25:26.001204 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:26.001219 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:26.001317 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:26.001677 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:26.001680 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:26.002163 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:26.002304 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:26.002313 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.003428 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.003431 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:26.004358 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:26.004369 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.005127 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.005130 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:26.005221 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.005790 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:26.005806 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:26.042052 (+ 36246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:26.048849 (+ 6797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:26.048852 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:26.048854 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:26.049099 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:26.049436 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:26.049439 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:26.049441 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:26.049541 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:26.050304 (+ 763us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:26.050308 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:26.050400 (+ 92us) client_negotiation.cc:769] Sending connection context
0808 08:25:26.051173 (+ 773us) client_negotiation.cc:240] Negotiation successful
0808 08:25:26.051290 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":89,"threads_started":1}
I20260808 08:25:26.053891 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.193:39011
--local_ip_for_outbound_sockets=127.22.235.193
--tserver_master_addrs=127.22.235.254:46527
--webserver_port=36801
--webserver_interface=127.22.235.193
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.193.keytab
--principal=oryx/127.22.235.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:36355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.22.235.254:46527,127.22.235.253:42359
--tserver_master_addrs=127.22.235.254:46527,127.22.235.253:42359,127.22.235.252:42345 with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 08:25:26 dist-test-slave-f7th krb5kdc[1076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177524, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.235.254@KRBTEST.COM for oryx/127.22.235.252@KRBTEST.COM
I20260808 08:25:26.062887 2375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.055833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46376 (local address 127.22.235.252:42345)
0808 08:25:26.055947 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:26.055951 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:26.056107 (+ 156us) server_negotiation.cc:407] Connection header received
0808 08:25:26.056166 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:26.056168 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:26.056210 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:26.056270 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:26.056681 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.057163 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.057906 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.058118 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.058203 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.060033 (+ 1830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:26.060037 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:26.060039 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:26.060060 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:26.061375 (+ 1315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:26.061796 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:26.061800 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:26.061801 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:26.061874 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:26.062125 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:26.062128 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:26.062130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:26.062302 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:26.062388 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:26.062646 (+ 258us) server_negotiation.cc:299] Negotiation successful
0808 08:25:26.062744 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 08:25:26.062887 2089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.055778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:42345 (local address 127.0.0.1:46376)
0808 08:25:26.055946 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:26.055960 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:26.056043 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:26.056308 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:26.056312 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:26.056474 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:26.056559 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:26.056564 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.057262 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.057266 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:26.057799 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:26.057807 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.058215 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.058219 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:26.058311 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.058716 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:26.058730 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:26.059905 (+ 1175us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:26.061486 (+ 1581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:26.061515 (+ 29us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:26.061518 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:26.061704 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:26.061959 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:26.061962 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:26.061964 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:26.062045 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:26.062392 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:26.062396 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:26.062495 (+ 99us) client_negotiation.cc:769] Sending connection context
0808 08:25:26.062582 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 08:25:26.062729 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103}
I20260808 08:25:26.064071 1157 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.22.235.252:42345: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558"
member_type: NON_VOTER
last_known_addr {
host: "127.22.235.252"
port: 42345
}
attrs {
promote: true
}
}
dest_uuid: "00ae12b3a32549bb9d8a3092fa6e0139"
cas_config_opid_index: 5
I20260808 08:25:26.064687 1157 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: NON_VOTER last_known_addr { host: "127.22.235.253" port: 42359 } attrs { promote: true } } peers { permanent_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558" member_type: NON_VOTER last_known_addr { host: "127.22.235.252" port: 42345 } attrs { promote: true } }
I20260808 08:25:26.065727 2095 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER b79f9c81ea0b4abeadd4ac1dbf54f558 (127.22.235.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: NON_VOTER last_known_addr { host: "127.22.235.253" port: 42359 } attrs { promote: true } } peers { permanent_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558" member_type: NON_VOTER last_known_addr { host: "127.22.235.252" port: 42345 } attrs { promote: true } } }
I20260808 08:25:26.066812 2320 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:26.067292 2095 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.22.235.252:42345
I20260808 08:25:26.067476 2095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: NON_VOTER last_known_addr { host: "127.22.235.253" port: 42359 } attrs { promote: true } } peers { permanent_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558" member_type: NON_VOTER last_known_addr { host: "127.22.235.252" port: 42345 } attrs { promote: true } } }
I20260808 08:25:26.067567 2095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 [sys.catalog]: This master's current role is: LEADER
I20260808 08:25:26.068284 2320 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558 [term 1 FOLLOWER]: Refusing update from remote peer 00ae12b3a32549bb9d8a3092fa6e0139: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 08:25:26.068287 2122 master.cc:537] Master@127.22.235.252:42345 shutting down...
I20260808 08:25:26.069284 2091 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558" member_type: NON_VOTER last_known_addr { host: "127.22.235.252" port: 42345 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 08:25:26.074038 1126 consensus_peers.cc:487] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 -> Peer b79f9c81ea0b4abeadd4ac1dbf54f558 (127.22.235.252:42345): Couldn't send request to peer b79f9c81ea0b4abeadd4ac1dbf54f558. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 08:25:26.074306 2089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.065970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.253:42359 (local address 127.0.0.1:51844)
0808 08:25:26.069136 (+ 3166us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:26.069153 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:26.069233 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:26.069780 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:26.069784 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:26.069802 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:25:26.069884 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:26.069890 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.071024 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.071027 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:26.072164 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:26.072175 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.073461 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.073476 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:26.073563 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.074045 (+ 482us) client_negotiation.cc:769] Sending connection context
0808 08:25:26.074116 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 08:25:26.074188 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3016}
I20260808 08:25:26.074922 2376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.066214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51844 (local address 127.22.235.253:42359)
0808 08:25:26.069404 (+ 3190us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:26.069408 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:26.069422 (+ 14us) server_negotiation.cc:407] Connection header received
0808 08:25:26.069481 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:26.069483 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:26.069546 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:26.069643 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:25:26.070013 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.070815 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.072303 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.073133 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.073247 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.073397 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:26.074727 (+ 1330us) server_negotiation.cc:299] Negotiation successful
0808 08:25:26.074804 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3094}
I20260808 08:25:26.075610 2189 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [term 1 LEARNER]: Refusing update from remote peer 00ae12b3a32549bb9d8a3092fa6e0139: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 08:25:26.075654 2122 raft_consensus.cc:2229] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 08:25:26.075934 2122 raft_consensus.cc:2258] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:25:26.076071 2122 tablet_replica.cc:330] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558: stopping tablet replica
I20260808 08:25:26.076086 2091 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: NON_VOTER last_known_addr { host: "127.22.235.253" port: 42359 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260808 08:25:26.077998 2189 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER b79f9c81ea0b4abeadd4ac1dbf54f558 (127.22.235.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: NON_VOTER last_known_addr { host: "127.22.235.253" port: 42359 } attrs { promote: true } } peers { permanent_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558" member_type: NON_VOTER last_known_addr { host: "127.22.235.252" port: 42345 } attrs { promote: true } } }
I20260808 08:25:26.079538 2413 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: NON_VOTER last_known_addr { host: "127.22.235.253" port: 42359 } attrs { promote: true } } peers { permanent_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558" member_type: NON_VOTER last_known_addr { host: "127.22.235.252" port: 42345 } attrs { promote: true } } }
I20260808 08:25:26.079685 2413 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [sys.catalog]: This master's current role is: LEARNER
I20260808 08:25:26.079998 2244 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00ae12b3a32549bb9d8a3092fa6e0139. Latest consensus state: current_term: 1 leader_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: NON_VOTER last_known_addr { host: "127.22.235.253" port: 42359 } attrs { promote: true } } peers { permanent_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558" member_type: NON_VOTER last_known_addr { host: "127.22.235.252" port: 42345 } attrs { promote: true } } }
I20260808 08:25:26.080083 2244 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [sys.catalog]: This master's current role is: LEARNER
I20260808 08:25:26.082330 2095 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139: attempting to promote NON_VOTER ad643f833e2a4adc89d9c6c143dbb971 to VOTER
I20260808 08:25:26.082720 2095 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 42359 } attrs { promote: false } } peers { permanent_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558" member_type: NON_VOTER last_known_addr { host: "127.22.235.252" port: 42345 } attrs { promote: true } }
I20260808 08:25:26.083776 2189 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [term 1 LEARNER]: Refusing update from remote peer 00ae12b3a32549bb9d8a3092fa6e0139: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260808 08:25:26.084160 2089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.083638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:42345 (local address 127.0.0.1:46378)
0808 08:25:26.084009 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:26.084085 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.252:42345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":293}
I20260808 08:25:26.084152 2090 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 42359 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260808 08:25:26.085094 2089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.084847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:42345 (local address 127.0.0.1:46386)
0808 08:25:26.084980 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:26.085030 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.252:42345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260808 08:25:26.085283 1126 consensus_peers.cc:487] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 -> Peer b79f9c81ea0b4abeadd4ac1dbf54f558 (127.22.235.252:42345): Couldn't send request to peer b79f9c81ea0b4abeadd4ac1dbf54f558. Status: Network error: Client connection negotiation failed: client connection to 127.22.235.252:42345: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 08:25:26.085856 2091 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, ad643f833e2a4adc89d9c6c143dbb971 (127.22.235.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 42359 } attrs { promote: false } } peers { permanent_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558" member_type: NON_VOTER last_known_addr { host: "127.22.235.252" port: 42345 } attrs { promote: true } } }
I20260808 08:25:26.087190 2091 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 42359 } attrs { promote: false } } peers { permanent_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558" member_type: NON_VOTER last_known_addr { host: "127.22.235.252" port: 42345 } attrs { promote: true } } }
I20260808 08:25:26.087296 2091 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 [sys.catalog]: This master's current role is: LEADER
I20260808 08:25:26.087445 2095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 42359 } attrs { promote: false } } peers { permanent_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558" member_type: NON_VOTER last_known_addr { host: "127.22.235.252" port: 42345 } attrs { promote: true } } }
I20260808 08:25:26.087539 2095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 [sys.catalog]: This master's current role is: LEADER
I20260808 08:25:26.088061 2189 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, ad643f833e2a4adc89d9c6c143dbb971 (127.22.235.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 42359 } attrs { promote: false } } peers { permanent_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558" member_type: NON_VOTER last_known_addr { host: "127.22.235.252" port: 42345 } attrs { promote: true } } }
I20260808 08:25:26.090063 2244 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 42359 } attrs { promote: false } } peers { permanent_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558" member_type: NON_VOTER last_known_addr { host: "127.22.235.252" port: 42345 } attrs { promote: true } } }
I20260808 08:25:26.090173 2244 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 08:25:26.093194 2122 master.cc:552] Master@127.22.235.252:42345 shutdown complete.
I20260808 08:25:26.093241 2122 master_runner.cc:305] Clearing existing system tablet
I20260808 08:25:26.093454 2122 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:26.093513 2122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:26.094543 2122 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 08:25:26.095315 2416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:26.095523 2122 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:26.095652 2122 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 08:25:26.095708 2122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-2/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-2/wal
uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558"
format_stamp: "Formatted at 2026-08-08 08:25:25 on dist-test-slave-f7th"
server_key: "b408d220b8d4132de2eb13587d4636d9"
server_key_iv: "1ff6fe6f978d2efd4459c6356a4a30a3"
server_key_version: "encryptionkey@0"
I20260808 08:25:26.097404 2122 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:25:26.099710 2122 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 08:25:26.099800 2122 log.cc:1196] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260808 08:25:26.100049 2122 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558: Deleting consensus metadata
I20260808 08:25:26.100203 2122 master_runner.cc:315] Copying system tablet from 127.22.235.254:46527
I20260808 08:25:26.101276 2122 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558: tablet copy: Beginning tablet copy session from remote peer at address 127.22.235.254:46527
I20260808 08:25:26.108356 1213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.101851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44748 (local address 127.22.235.254:46527)
0808 08:25:26.101972 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:26.101977 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:26.102152 (+ 175us) server_negotiation.cc:407] Connection header received
0808 08:25:26.102268 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:26.102271 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:26.102319 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:26.102387 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:26.102819 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.103338 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.104131 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.104356 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.104442 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.105517 (+ 1075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:26.105521 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:26.105524 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:26.105548 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:26.106854 (+ 1306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:26.107259 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:26.107262 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:26.107263 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:26.107339 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:26.107587 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:26.107590 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:26.107592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:26.107760 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:26.107847 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:26.108111 (+ 264us) server_negotiation.cc:299] Negotiation successful
0808 08:25:26.108216 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 08:25:26.108371 2425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.101775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46527 (local address 127.0.0.1:44748)
0808 08:25:26.102070 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:26.102081 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:26.102188 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:26.102415 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:26.102418 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:26.102609 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:26.102705 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:26.102711 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.103441 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.103445 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:26.104016 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:26.104025 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.104457 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.104461 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:26.104557 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.105045 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:26.105060 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:26.105366 (+ 306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:26.106959 (+ 1593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:26.106962 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:26.106972 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:26.107169 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:26.107422 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:26.107425 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:26.107427 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:26.107507 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:26.107853 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:26.107856 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:26.107951 (+ 95us) client_negotiation.cc:769] Sending connection context
0808 08:25:26.108047 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 08:25:26.108188 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":83,"threads_started":1}
I20260808 08:25:26.109119 1190 tablet_copy_service.cc:140] P 00ae12b3a32549bb9d8a3092fa6e0139: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer b79f9c81ea0b4abeadd4ac1dbf54f558 ({username='oryx', principal='oryx/127.22.235.252@KRBTEST.COM'} at 127.0.0.1:44748)
I20260808 08:25:26.109215 1190 tablet_copy_service.cc:161] P 00ae12b3a32549bb9d8a3092fa6e0139: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer b79f9c81ea0b4abeadd4ac1dbf54f558 at {username='oryx', principal='oryx/127.22.235.252@KRBTEST.COM'} at 127.0.0.1:44748: session id = b79f9c81ea0b4abeadd4ac1dbf54f558-00000000000000000000000000000000
I20260808 08:25:26.110074 1190 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 08:25:26.111145 2122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 08:25:26.112906 2122 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558: tablet copy: Starting download of 0 data blocks...
I20260808 08:25:26.113090 2122 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558: tablet copy: Starting download of 1 WAL segments...
I20260808 08:25:26.114357 2122 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 08:25:26.115782 1190 tablet_copy_service.cc:342] P 00ae12b3a32549bb9d8a3092fa6e0139: Request end of tablet copy session b79f9c81ea0b4abeadd4ac1dbf54f558-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.22.235.252@KRBTEST.COM'} at 127.0.0.1:44748
I20260808 08:25:26.115854 1190 tablet_copy_service.cc:433] P 00ae12b3a32549bb9d8a3092fa6e0139: ending tablet copy session b79f9c81ea0b4abeadd4ac1dbf54f558-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer b79f9c81ea0b4abeadd4ac1dbf54f558
I20260808 08:25:26.122167 2122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:26.123706 2432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:26.123934 2429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:26.123999 2428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:26.124420 2122 server_base.cc:1034] running on GCE node
I20260808 08:25:26.124578 2122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:26.124872 2122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:26.126003 2122 hybrid_clock.cc:648] HybridClock initialized: now 1786177526125996 us; error 25 us; skew 500 ppm
Aug 08 08:25:26 dist-test-slave-f7th krb5kdc[1076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177526, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.235.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:26.127348 2122 init.cc:377] Logged in from keytab as oryx/127.22.235.252@KRBTEST.COM (short username oryx)
I20260808 08:25:26.127764 2122 webserver.cc:460] Webserver started at http://127.22.235.252:34129/ using document root <none> and password file <none>
I20260808 08:25:26.127945 2122 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:26.127981 2122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:26.129091 2122 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 08:25:26.129999 2443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:26.130218 2122 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 08:25:26.130278 2122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-2/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-2/wal
uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558"
format_stamp: "Formatted at 2026-08-08 08:25:25 on dist-test-slave-f7th"
server_key: "b408d220b8d4132de2eb13587d4636d9"
server_key_iv: "1ff6fe6f978d2efd4459c6356a4a30a3"
server_key_version: "encryptionkey@0"
I20260808 08:25:26.130358 2122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:26.158648 2122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:26.160512 2122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:26.165129 2122 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.252:42345
I20260808 08:25:26.165725 2122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/master-2/data/info.pb
I20260808 08:25:26.167723 2541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.252:42345 every 8 connection(s)
I20260808 08:25:26.169108 2542 sys_catalog.cc:263] Verifying existing consensus state
I20260808 08:25:26.169994 2542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558: Bootstrap starting.
I20260808 08:25:26.178791 2542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 08:25:26.179054 2542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558: Bootstrap complete.
I20260808 08:25:26.179534 2542 raft_consensus.cc:348] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 42359 } attrs { promote: false } } peers { permanent_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558" member_type: NON_VOTER last_known_addr { host: "127.22.235.252" port: 42345 } attrs { promote: true } }
I20260808 08:25:26.179649 2542 raft_consensus.cc:729] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b79f9c81ea0b4abeadd4ac1dbf54f558, State: Initialized, Role: LEARNER
I20260808 08:25:26.179754 2542 consensus_queue.cc:260] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 42359 } attrs { promote: false } } peers { permanent_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558" member_type: NON_VOTER last_known_addr { host: "127.22.235.252" port: 42345 } attrs { promote: true } }
I20260808 08:25:26.180064 2547 sys_catalog.cc:455] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 42359 } attrs { promote: false } } peers { permanent_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558" member_type: NON_VOTER last_known_addr { host: "127.22.235.252" port: 42345 } attrs { promote: true } } }
I20260808 08:25:26.180209 2542 sys_catalog.cc:564] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:25:26.180235 2547 sys_catalog.cc:458] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558 [sys.catalog]: This master's current role is: LEARNER
I20260808 08:25:26.183228 2562 catalog_manager.cc:1259] Loaded cluster ID: aafd38e6ed664f44b830a3f06aac46a6
I20260808 08:25:26.183346 2562 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558: loading cluster ID for follower catalog manager: success
I20260808 08:25:26.185411 2562 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558: acquiring CA information for follower catalog manager: success
I20260808 08:25:26.185547 1696 heartbeater.cc:503] Master 127.22.235.254:46527 was elected leader, sending a full tablet report...
I20260808 08:25:26.187104 2562 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 08:25:26.205236 2411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:26.205471 2411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:26.205510 2411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:26.209352 2411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:26.209399 2411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:26.209452 2411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.193
I20260808 08:25:26.214388 2411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:36355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.193.keytab
--principal=oryx/127.22.235.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.193:39011
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.235.193
--webserver_port=36801
--tserver_master_addrs=127.22.235.254:46527,127.22.235.253:42359,127.22.235.252:42345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.193
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:26.215301 2411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:26.216249 2411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:26.225382 2576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:26.225435 2577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:26.225383 2579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:26.225790 2411 server_base.cc:1034] running on GCE node
I20260808 08:25:26.226178 2411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:26.226763 2411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:26.227933 2411 hybrid_clock.cc:648] HybridClock initialized: now 1786177526227908 us; error 34 us; skew 500 ppm
Aug 08 08:25:26 dist-test-slave-f7th krb5kdc[1076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177526, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.235.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:26.230588 2411 init.cc:377] Logged in from keytab as oryx/127.22.235.193@KRBTEST.COM (short username oryx)
I20260808 08:25:26.231503 2411 webserver.cc:460] Webserver started at http://127.22.235.193:36801/ using document root <none> and password file <none>
I20260808 08:25:26.232031 2411 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:26.232103 2411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:26.235939 2411 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 08:25:26.238023 2588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:26.239209 2411 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 08:25:26.239311 2411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-0/wal
uuid: "11de37660d0946beb33ae725e791d907"
format_stamp: "Formatted at 2026-08-08 08:25:24 on dist-test-slave-f7th"
server_key: "4d6a72b58c789a36984279c4545d821e"
server_key_iv: "bdb93fedea4f81c3e18169357cc7d0be"
server_key_version: "encryptionkey@0"
I20260808 08:25:26.239827 2411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:26.251500 2411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:26.254647 2411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:26.254833 2411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:26.255548 2411 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:25:26.256425 2411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:25:26.256489 2411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:26.256552 2411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:25:26.256582 2411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:26.264905 2411 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.193:39011
I20260808 08:25:26.264952 2789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.193:39011 every 8 connection(s)
I20260808 08:25:26.266088 2411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
I20260808 08:25:26.272065 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 2411
I20260808 08:25:26.272233 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 1485
Aug 08 08:25:26 dist-test-slave-f7th krb5kdc[1076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177526, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.235.193@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM
Aug 08 08:25:26 dist-test-slave-f7th krb5kdc[1076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177526, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.235.193@KRBTEST.COM for oryx/127.22.235.253@KRBTEST.COM
Aug 08 08:25:26 dist-test-slave-f7th krb5kdc[1076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177526, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.235.193@KRBTEST.COM for oryx/127.22.235.252@KRBTEST.COM
I20260808 08:25:26.287526 1213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.267828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:38741 (local address 127.22.235.254:46527)
0808 08:25:26.267963 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:26.267967 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:26.268779 (+ 812us) server_negotiation.cc:407] Connection header received
0808 08:25:26.270335 (+ 1556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:26.270338 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:26.270390 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:26.270439 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:26.272689 (+ 2250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.273304 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.276904 (+ 3600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.277204 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.277269 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.281001 (+ 3732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:26.281005 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:26.281007 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:26.281033 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:26.284591 (+ 3558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:26.285371 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:26.285375 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:26.285377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:26.285475 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:26.286284 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:26.286288 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:26.286290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:26.286471 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:26.286563 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:26.287239 (+ 676us) server_negotiation.cc:299] Negotiation successful
0808 08:25:26.287359 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 08:25:26.288482 2797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.268191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46527 (local address 127.22.235.193:38741)
0808 08:25:26.268601 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:26.268676 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:26.270092 (+ 1416us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:26.270636 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:26.270644 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:26.271101 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:26.271615 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:26.271630 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.274958 (+ 3328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.274962 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:26.276769 (+ 1807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:26.276781 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.277439 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.277443 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:26.277614 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.279106 (+ 1492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:26.279126 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:26.280826 (+ 1700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:26.284773 (+ 3947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:26.284781 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:26.284800 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:26.285244 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:26.286001 (+ 757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:26.286003 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:26.286004 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:26.286187 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:26.286615 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:26.286620 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:26.286965 (+ 345us) client_negotiation.cc:769] Sending connection context
0808 08:25:26.287208 (+ 243us) client_negotiation.cc:240] Negotiation successful
0808 08:25:26.287544 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":64,"spinlock_wait_cycles":11648,"thread_start_us":110,"threads_started":1}
I20260808 08:25:26.289749 2791 heartbeater.cc:347] Connected to a master server at 127.22.235.254:46527
I20260808 08:25:26.290010 2791 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:26.290603 2791 heartbeater.cc:511] Master 127.22.235.254:46527 requested a full tablet report, sending...
I20260808 08:25:26.293025 1157 ts_manager.cc:194] Re-registered known tserver with Master: 11de37660d0946beb33ae725e791d907 (127.22.235.193:39011)
I20260808 08:25:26.293318 2376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.269263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:54699 (local address 127.22.235.253:42359)
0808 08:25:26.269390 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:26.269394 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:26.269503 (+ 109us) server_negotiation.cc:407] Connection header received
0808 08:25:26.270335 (+ 832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:26.270338 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:26.270390 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:26.270439 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:26.271784 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.272545 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.282465 (+ 9920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.282761 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.282824 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.285494 (+ 2670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:26.285498 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:26.285501 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:26.285532 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:26.290813 (+ 5281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:26.291459 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:26.291476 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:26.291479 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:26.291593 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:26.291969 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:26.291972 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:26.291974 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:26.292243 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:26.292327 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:26.293000 (+ 673us) server_negotiation.cc:299] Negotiation successful
0808 08:25:26.293147 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 08:25:26.293710 1157 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.235.193@KRBTEST.COM'} at 127.22.235.193:38741
I20260808 08:25:26.294163 2799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.269116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.253:42359 (local address 127.22.235.193:54699)
0808 08:25:26.269407 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:26.269421 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:26.270092 (+ 671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:26.270636 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:26.270644 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:26.271127 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:26.271615 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:26.271630 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.278974 (+ 7344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.278977 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:26.282322 (+ 3345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:26.282334 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.282953 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.282955 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:26.283026 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.283978 (+ 952us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:26.283999 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:26.285295 (+ 1296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:26.291028 (+ 5733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:26.291031 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:26.291034 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:26.291320 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:26.291731 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:26.291735 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:26.291738 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:26.291844 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:26.292640 (+ 796us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:26.292648 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:26.292765 (+ 117us) client_negotiation.cc:769] Sending connection context
0808 08:25:26.293707 (+ 942us) client_negotiation.cc:240] Negotiation successful
0808 08:25:26.293933 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"mutex_wait_us":61,"spinlock_wait_cycles":34176,"thread_start_us":91,"threads_started":1}
I20260808 08:25:26.295143 2804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.269856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:60073 (local address 127.22.235.252:42345)
0808 08:25:26.270115 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:26.270120 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:26.270135 (+ 15us) server_negotiation.cc:407] Connection header received
0808 08:25:26.270298 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:26.270301 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:26.270358 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:26.270440 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:26.271751 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.282955 (+ 11204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.283784 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.284120 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.284584 (+ 464us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.287255 (+ 2671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:26.287260 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:26.287263 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:26.287297 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:26.290500 (+ 3203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:26.293812 (+ 3312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:26.293815 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:26.293817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:26.293901 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:26.294211 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:26.294214 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:26.294215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:26.294426 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:26.294802 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:26.294874 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 08:25:26.294994 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":87,"threads_started":1}
I20260808 08:25:26.296006 2802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.269444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:42345 (local address 127.22.235.193:60073)
0808 08:25:26.269672 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:26.269687 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:26.270100 (+ 413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:26.270635 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:26.270642 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:26.271091 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:26.271613 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:26.271627 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.283081 (+ 11454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.283084 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:26.283666 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:26.283676 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.284258 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.284262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:26.284353 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.285574 (+ 1221us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:26.285586 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:26.287083 (+ 1497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:26.293454 (+ 6371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:26.293457 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:26.293459 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:26.293686 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:26.294005 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:26.294009 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:26.294011 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:26.294098 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:26.294551 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:26.294554 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:26.294630 (+ 76us) client_negotiation.cc:769] Sending connection context
0808 08:25:26.294706 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 08:25:26.295881 (+ 1175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":1067,"thread_start_us":106,"threads_started":1}
I20260808 08:25:26.297036 2790 heartbeater.cc:347] Connected to a master server at 127.22.235.252:42345
I20260808 08:25:26.297127 2790 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:26.297304 2790 heartbeater.cc:511] Master 127.22.235.252:42345 requested a full tablet report, sending...
I20260808 08:25:26.298462 2455 ts_manager.cc:194] Registered new tserver with Master: 11de37660d0946beb33ae725e791d907 (127.22.235.193:39011)
I20260808 08:25:26.298974 2793 heartbeater.cc:347] Connected to a master server at 127.22.235.253:42359
I20260808 08:25:26.299072 2793 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:26.299198 2793 heartbeater.cc:511] Master 127.22.235.253:42359 requested a full tablet report, sending...
I20260808 08:25:26.300189 2169 ts_manager.cc:194] Registered new tserver with Master: 11de37660d0946beb33ae725e791d907 (127.22.235.193:39011)
I20260808 08:25:26.305225 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.194:39211
--local_ip_for_outbound_sockets=127.22.235.194
--tserver_master_addrs=127.22.235.254:46527
--webserver_port=36631
--webserver_interface=127.22.235.194
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.194.keytab
--principal=oryx/127.22.235.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:36355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.22.235.254:46527,127.22.235.253:42359
--tserver_master_addrs=127.22.235.254:46527,127.22.235.253:42359,127.22.235.252:42345 with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 08:25:26.407879 1916 heartbeater.cc:503] Master 127.22.235.254:46527 was elected leader, sending a full tablet report...
W20260808 08:25:26.417362 2806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:26.417624 2806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:26.417666 2806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:26.421572 2806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:26.421622 2806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:26.421697 2806 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.194
I20260808 08:25:26.426692 2806 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:36355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.194.keytab
--principal=oryx/127.22.235.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.194:39211
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.235.194
--webserver_port=36631
--tserver_master_addrs=127.22.235.254:46527,127.22.235.253:42359,127.22.235.252:42345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2806
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.194
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:26.427637 2806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:26.428609 2806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:26.437963 2818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:26.437992 2815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:26.437994 2814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:26.438476 2806 server_base.cc:1034] running on GCE node
I20260808 08:25:26.438845 2806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:26.439442 2806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:26.440598 2806 hybrid_clock.cc:648] HybridClock initialized: now 1786177526440582 us; error 26 us; skew 500 ppm
Aug 08 08:25:26 dist-test-slave-f7th krb5kdc[1076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177526, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.235.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:26.443169 2806 init.cc:377] Logged in from keytab as oryx/127.22.235.194@KRBTEST.COM (short username oryx)
I20260808 08:25:26.444010 2806 webserver.cc:460] Webserver started at http://127.22.235.194:36631/ using document root <none> and password file <none>
I20260808 08:25:26.444535 2806 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:26.444608 2806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:26.448241 2806 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:25:26.450209 2827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:26.451448 2806 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 08:25:26.451558 2806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-1/wal
uuid: "6b3a62b7e6a249d197410707a5bc428a"
format_stamp: "Formatted at 2026-08-08 08:25:25 on dist-test-slave-f7th"
server_key: "a94874ade4d471b354557b00af091e93"
server_key_iv: "ab19031d75155b0ce583ae6bc7d4db7c"
server_key_version: "encryptionkey@0"
I20260808 08:25:26.452068 2806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:26.466187 2806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:26.468878 2806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:26.469048 2806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:26.469574 2806 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:25:26.470388 2806 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:25:26.470433 2806 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:26.470475 2806 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:25:26.470505 2806 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:26.478921 2806 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.194:39211
I20260808 08:25:26.478981 3005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.194:39211 every 8 connection(s)
I20260808 08:25:26.479861 2806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
I20260808 08:25:26.481170 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 2806
I20260808 08:25:26.481333 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 1704
I20260808 08:25:26.516676 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.195:40217
--local_ip_for_outbound_sockets=127.22.235.195
--tserver_master_addrs=127.22.235.254:46527
--webserver_port=34531
--webserver_interface=127.22.235.195
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.195.keytab
--principal=oryx/127.22.235.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:36355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.22.235.254:46527,127.22.235.253:42359
--tserver_master_addrs=127.22.235.254:46527,127.22.235.253:42359,127.22.235.252:42345 with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 08:25:26 dist-test-slave-f7th krb5kdc[1076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177526, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.235.194@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM
Aug 08 08:25:26 dist-test-slave-f7th krb5kdc[1076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177526, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.235.194@KRBTEST.COM for oryx/127.22.235.252@KRBTEST.COM
Aug 08 08:25:26 dist-test-slave-f7th krb5kdc[1076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177526, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.235.194@KRBTEST.COM for oryx/127.22.235.253@KRBTEST.COM
I20260808 08:25:26.524852 2376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.504406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:52519 (local address 127.22.235.253:42359)
0808 08:25:26.504598 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:26.504603 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:26.509841 (+ 5238us) server_negotiation.cc:407] Connection header received
0808 08:25:26.511662 (+ 1821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:26.511665 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:26.511713 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:26.511762 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:26.513713 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.514519 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.516269 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.516662 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.516725 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.521187 (+ 4462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:26.521193 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:26.521196 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:26.521228 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:26.522954 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:26.523388 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:26.523392 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:26.523393 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:26.523477 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:26.523731 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:26.523734 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:26.523736 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:26.523915 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:26.523998 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:26.524540 (+ 542us) server_negotiation.cc:299] Negotiation successful
0808 08:25:26.524683 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 08:25:26.524881 2804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.510430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:60939 (local address 127.22.235.252:42345)
0808 08:25:26.510636 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:26.510642 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:26.510958 (+ 316us) server_negotiation.cc:407] Connection header received
0808 08:25:26.511772 (+ 814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:26.511775 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:26.511833 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:26.511896 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:26.513710 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.514468 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.516256 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.516595 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.516725 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.520420 (+ 3695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:26.520424 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:26.520427 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:26.520455 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:26.522478 (+ 2023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:26.523081 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:26.523085 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:26.523087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:26.523180 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:26.523596 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:26.523600 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:26.523602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:26.523830 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:26.523925 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:26.524557 (+ 632us) server_negotiation.cc:299] Negotiation successful
0808 08:25:26.524720 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260808 08:25:26.525900 3014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.510562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:42345 (local address 127.22.235.194:60939)
0808 08:25:26.510876 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:26.510889 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:26.511436 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:26.512102 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:26.512112 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:26.512753 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:26.513559 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:26.513570 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.514631 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.514634 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:26.516120 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:26.516131 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.516914 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.516918 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:26.517177 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.517920 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:26.517941 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:26.520235 (+ 2294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:26.522607 (+ 2372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:26.522610 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:26.522612 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:26.522952 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:26.523286 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:26.523289 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:26.523291 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:26.523483 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:26.523982 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:26.523989 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:26.524289 (+ 300us) client_negotiation.cc:769] Sending connection context
0808 08:25:26.525094 (+ 805us) client_negotiation.cc:240] Negotiation successful
0808 08:25:26.525235 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"mutex_wait_us":300,"spinlock_wait_cycles":35840,"thread_start_us":94,"threads_started":1}
I20260808 08:25:26.526154 3012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.482185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.253:42359 (local address 127.22.235.194:52519)
0808 08:25:26.509663 (+ 27478us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:26.509742 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:26.511432 (+ 1690us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:26.512102 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:26.512112 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:26.512744 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:26.513548 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:26.513568 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.514661 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.514664 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:26.516134 (+ 1470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:26.516145 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.516914 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.516918 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:26.517286 (+ 368us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.517965 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:26.517981 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:26.520540 (+ 2559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:26.523066 (+ 2526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:26.523069 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:26.523070 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:26.523306 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:26.523561 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:26.523564 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:26.523566 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:26.523643 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:26.524009 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:26.524012 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:26.524278 (+ 266us) client_negotiation.cc:769] Sending connection context
0808 08:25:26.524454 (+ 176us) client_negotiation.cc:240] Negotiation successful
0808 08:25:26.524816 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27177,"mutex_wait_us":360,"spinlock_wait_cycles":24576,"thread_start_us":27011,"threads_started":1}
I20260808 08:25:26.527295 3006 heartbeater.cc:347] Connected to a master server at 127.22.235.252:42345
I20260808 08:25:26.527493 3006 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:26.527822 3015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.511144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46527 (local address 127.22.235.194:51015)
0808 08:25:26.511377 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:26.511389 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:26.511519 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:26.512102 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:26.512112 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:26.512744 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:26.513548 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:26.513569 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.514661 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.514664 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:26.516135 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:26.516145 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.516914 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.516918 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:26.517176 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.518065 (+ 889us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:26.518078 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:26.520543 (+ 2465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:26.522520 (+ 1977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:26.522527 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:26.522545 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:26.522947 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:26.523288 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:26.523290 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:26.523292 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:26.526856 (+ 3564us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:26.527317 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:26.527321 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:26.527417 (+ 96us) client_negotiation.cc:769] Sending connection context
0808 08:25:26.527522 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 08:25:26.527654 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":375,"spinlock_wait_cycles":3968,"thread_start_us":85,"threads_started":1}
I20260808 08:25:26.527872 1213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.511030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:51015 (local address 127.22.235.254:46527)
0808 08:25:26.511148 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:26.511152 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:26.511459 (+ 307us) server_negotiation.cc:407] Connection header received
0808 08:25:26.511661 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:26.511664 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:26.511713 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:26.511762 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:26.513713 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.514519 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.516269 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.516608 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.516704 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.520722 (+ 4018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:26.520726 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:26.520728 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:26.520758 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:26.522370 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:26.523066 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:26.523070 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:26.523072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:26.523171 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:26.526990 (+ 3819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:26.526994 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:26.526995 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:26.527210 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:26.527315 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:26.527616 (+ 301us) server_negotiation.cc:299] Negotiation successful
0808 08:25:26.527733 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 08:25:26.528083 3006 heartbeater.cc:511] Master 127.22.235.252:42345 requested a full tablet report, sending...
I20260808 08:25:26.528967 3009 heartbeater.cc:347] Connected to a master server at 127.22.235.253:42359
I20260808 08:25:26.529059 3009 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:26.529218 3009 heartbeater.cc:511] Master 127.22.235.253:42359 requested a full tablet report, sending...
I20260808 08:25:26.529387 2455 ts_manager.cc:194] Registered new tserver with Master: 6b3a62b7e6a249d197410707a5bc428a (127.22.235.194:39211)
I20260808 08:25:26.529865 2169 ts_manager.cc:194] Registered new tserver with Master: 6b3a62b7e6a249d197410707a5bc428a (127.22.235.194:39211)
I20260808 08:25:26.530144 3007 heartbeater.cc:347] Connected to a master server at 127.22.235.254:46527
I20260808 08:25:26.530231 3007 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:26.530382 3007 heartbeater.cc:511] Master 127.22.235.254:46527 requested a full tablet report, sending...
I20260808 08:25:26.531037 1157 ts_manager.cc:194] Re-registered known tserver with Master: 6b3a62b7e6a249d197410707a5bc428a (127.22.235.194:39211)
I20260808 08:25:26.531796 1157 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.235.194@KRBTEST.COM'} at 127.22.235.194:51015
W20260808 08:25:26.631421 3016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:26.631667 3016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:26.631707 3016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:26.635730 3016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:26.635779 3016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:26.635834 3016 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.195
I20260808 08:25:26.640952 3016 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:36355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.195.keytab
--principal=oryx/127.22.235.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.195:40217
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.235.195
--webserver_port=34531
--tserver_master_addrs=127.22.235.254:46527,127.22.235.253:42359,127.22.235.252:42345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.195
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:26.641888 3016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:26.642829 3016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:26.652467 3026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:26.652436 3023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:26.652443 3029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:26.652601 3016 server_base.cc:1034] running on GCE node
I20260808 08:25:26.653188 3016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:26.653784 3016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:26.654930 3016 hybrid_clock.cc:648] HybridClock initialized: now 1786177526654909 us; error 28 us; skew 500 ppm
Aug 08 08:25:26 dist-test-slave-f7th krb5kdc[1076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177526, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.235.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:26.657575 3016 init.cc:377] Logged in from keytab as oryx/127.22.235.195@KRBTEST.COM (short username oryx)
I20260808 08:25:26.658337 3016 webserver.cc:460] Webserver started at http://127.22.235.195:34531/ using document root <none> and password file <none>
I20260808 08:25:26.658845 3016 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:26.658915 3016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:26.662633 3016 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:25:26.664646 3043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:26.665894 3016 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:25:26.666003 3016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-2/wal
uuid: "27bb37319a6742ccb01e572b068b4bc9"
format_stamp: "Formatted at 2026-08-08 08:25:25 on dist-test-slave-f7th"
server_key: "32a9527ec83350fb4f36147745b40f02"
server_key_iv: "a37af5a1022b8dd6b463555289399548"
server_key_version: "encryptionkey@0"
I20260808 08:25:26.666658 3016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:26.669360 3044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.665875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:42345 (local address 127.0.0.1:46402)
0808 08:25:26.666168 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:26.666180 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:26.666288 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:26.666560 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:26.666562 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:26.666575 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:25:26.666681 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:26.666688 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.667485 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.667488 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:26.668251 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:26.668261 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.668940 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.668944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:26.669035 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.669061 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 08:25:26.669142 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 08:25:26.669231 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":64,"threads_started":1}
I20260808 08:25:26.669356 2804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.665984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46402 (local address 127.22.235.252:42345)
0808 08:25:26.666137 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:26.666142 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:26.666254 (+ 112us) server_negotiation.cc:407] Connection header received
0808 08:25:26.666377 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:26.666381 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:26.666450 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:26.666560 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:25:26.666803 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.667368 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.668370 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.668831 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.668939 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.669082 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:26.669160 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 08:25:26.669229 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 08:25:26.670063 2494 raft_consensus.cc:1203] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260808 08:25:26.670482 2547 sys_catalog.cc:455] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00ae12b3a32549bb9d8a3092fa6e0139. Latest consensus state: current_term: 1 leader_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 42359 } attrs { promote: false } } peers { permanent_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558" member_type: NON_VOTER last_known_addr { host: "127.22.235.252" port: 42345 } attrs { promote: true } } }
I20260808 08:25:26.670619 2547 sys_catalog.cc:458] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558 [sys.catalog]: This master's current role is: LEARNER
I20260808 08:25:26.698417 3016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:26.701570 3016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:26.701735 3016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:26.702304 3016 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:25:26.703162 3016 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:25:26.703224 3016 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:26.703290 3016 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:25:26.703317 3016 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:26.711625 3016 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.195:40217
I20260808 08:25:26.711655 3242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.195:40217 every 8 connection(s)
I20260808 08:25:26.712556 3016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
I20260808 08:25:26.713581 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 3016
Aug 08 08:25:26 dist-test-slave-f7th krb5kdc[1076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177526, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.235.195@KRBTEST.COM for oryx/127.22.235.252@KRBTEST.COM
I20260808 08:25:26.726213 2376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.716657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51848 (local address 127.22.235.253:42359)
0808 08:25:26.716765 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:26.716769 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:26.717237 (+ 468us) server_negotiation.cc:407] Connection header received
0808 08:25:26.717335 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:26.717338 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:26.717390 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:26.717468 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:26.718308 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.719041 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.720314 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.720650 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.720720 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.722405 (+ 1685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:26.722410 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:26.722412 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:26.722435 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:26.724205 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:26.724731 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:26.724735 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:26.724737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:26.724821 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:26.725214 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:26.725218 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:26.725219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:26.725432 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:26.725551 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:26.725917 (+ 366us) server_negotiation.cc:299] Negotiation successful
0808 08:25:26.726035 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 08:25:26.727455 1213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.715481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44756 (local address 127.22.235.254:46527)
0808 08:25:26.715596 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:26.715601 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:26.715849 (+ 248us) server_negotiation.cc:407] Connection header received
0808 08:25:26.716342 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:26.716345 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:26.716402 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:26.716456 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:26.717830 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.718598 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.719756 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.720106 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.720178 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.723285 (+ 3107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:26.723289 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:26.723291 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:26.723319 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:26.725154 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:26.725575 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:26.725578 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:26.725580 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:26.725905 (+ 325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:26.726524 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:26.726528 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:26.726529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:26.726734 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:26.726853 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:26.727208 (+ 355us) server_negotiation.cc:299] Negotiation successful
0808 08:25:26.727311 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
Aug 08 08:25:26 dist-test-slave-f7th krb5kdc[1076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177526, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.235.195@KRBTEST.COM for oryx/127.22.235.253@KRBTEST.COM
I20260808 08:25:26.730145 3273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.716763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46418 (local address 127.22.235.252:42345)
0808 08:25:26.718017 (+ 1254us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:26.718021 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:26.718032 (+ 11us) server_negotiation.cc:407] Connection header received
0808 08:25:26.718061 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:26.718063 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:26.718099 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:26.718177 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:26.719084 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.719822 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.721289 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.721637 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.721727 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.723392 (+ 1665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:26.723395 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:26.723398 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:26.723419 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:26.727815 (+ 4396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:26.728474 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:26.728478 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:26.728479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:26.728606 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:26.728909 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:26.728912 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:26.728914 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:26.729118 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:26.729225 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:26.729776 (+ 551us) server_negotiation.cc:299] Negotiation successful
0808 08:25:26.729894 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1178,"thread_start_us":89,"threads_started":1}
I20260808 08:25:26.730311 2804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.714535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:47467 (local address 127.22.235.252:42345)
0808 08:25:26.714695 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:26.714700 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:26.715731 (+ 1031us) server_negotiation.cc:407] Connection header received
0808 08:25:26.717648 (+ 1917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:26.717652 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:26.717708 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:26.717761 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:26.719529 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.720201 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.721844 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.722139 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.722209 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.725387 (+ 3178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:26.725391 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:26.725394 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:26.725420 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:26.726903 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:26.728691 (+ 1788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:26.728694 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:26.728696 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:26.728776 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:26.729171 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:26.729174 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:26.729175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:26.729367 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:26.729445 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:26.730095 (+ 650us) server_negotiation.cc:299] Negotiation successful
0808 08:25:26.730198 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 08:25:26.732055 3261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.715028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:42345 (local address 127.22.235.195:47467)
0808 08:25:26.715576 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:26.715632 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:26.717360 (+ 1728us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:26.718158 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:26.718167 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:26.718700 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:26.719359 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:26.719378 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.720354 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.720357 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:26.721725 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:26.721735 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.722378 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.722382 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:26.722589 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.723364 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:26.723384 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:26.725218 (+ 1834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:26.727015 (+ 1797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:26.727021 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:26.727036 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:26.728563 (+ 1527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:26.728883 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:26.728886 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:26.728887 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:26.729058 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:26.729524 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:26.729530 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:26.729822 (+ 292us) client_negotiation.cc:769] Sending connection context
0808 08:25:26.730495 (+ 673us) client_negotiation.cc:240] Negotiation successful
0808 08:25:26.730823 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":143,"threads_started":1}
I20260808 08:25:26.733243 3243 heartbeater.cc:347] Connected to a master server at 127.22.235.252:42345
I20260808 08:25:26.733505 3243 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:26.730728 1157 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44756:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
Aug 08 08:25:26 dist-test-slave-f7th krb5kdc[1076](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177526, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.235.195@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM
I20260808 08:25:26.734078 3243 heartbeater.cc:511] Master 127.22.235.252:42345 requested a full tablet report, sending...
W20260808 08:25:26.734899 1157 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 08:25:26.735165 2455 ts_manager.cc:194] Registered new tserver with Master: 27bb37319a6742ccb01e572b068b4bc9 (127.22.235.195:40217)
I20260808 08:25:26.735318 3272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.716672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:39679 (local address 127.22.235.253:42359)
0808 08:25:26.716892 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:26.716895 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:26.722334 (+ 5439us) server_negotiation.cc:407] Connection header received
0808 08:25:26.722373 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:26.722376 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:26.722420 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:26.722496 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:26.724428 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.725211 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.727909 (+ 2698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.728260 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.728355 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.730805 (+ 2450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:26.730808 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:26.730811 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:26.730834 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:26.732870 (+ 2036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:26.733378 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:26.733381 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:26.733383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:26.733464 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:26.734020 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:26.734023 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:26.734025 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:26.734275 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:26.734364 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:26.735047 (+ 683us) server_negotiation.cc:299] Negotiation successful
0808 08:25:26.735166 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":164,"thread_start_us":68,"threads_started":1}
I20260808 08:25:26.736312 3271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.716440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.253:42359 (local address 127.22.235.195:39679)
0808 08:25:26.720510 (+ 4070us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:26.720523 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:26.720653 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:26.722610 (+ 1957us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:26.722613 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:26.723769 (+ 1156us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:26.723913 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:26.723920 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.726957 (+ 3037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.726960 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:26.727774 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:26.727786 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.728360 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.728364 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:26.728451 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.728904 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:26.728919 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:26.730658 (+ 1739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:26.732983 (+ 2325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:26.732986 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:26.732988 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:26.733222 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:26.733619 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:26.733622 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:26.733624 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:26.733717 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:26.734408 (+ 691us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:26.734411 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:26.734510 (+ 99us) client_negotiation.cc:769] Sending connection context
0808 08:25:26.734607 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 08:25:26.735467 (+ 860us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4009,"mutex_wait_us":3,"thread_start_us":92,"threads_started":1}
I20260808 08:25:26.737118 3245 heartbeater.cc:347] Connected to a master server at 127.22.235.253:42359
I20260808 08:25:26.737198 3245 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:26.737362 3245 heartbeater.cc:511] Master 127.22.235.253:42359 requested a full tablet report, sending...
I20260808 08:25:26.738205 3264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.716057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46527 (local address 127.22.235.195:33539)
0808 08:25:26.720272 (+ 4215us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:26.720285 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:26.722168 (+ 1883us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:26.722915 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:26.722918 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:26.723470 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:26.725626 (+ 2156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:26.725636 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.726666 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.726669 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:26.731527 (+ 4858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:26.731539 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.732230 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.732234 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:26.732318 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.732867 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:26.732880 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:26.734310 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:26.736699 (+ 2389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:26.736702 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:26.736704 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:26.736917 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:26.737207 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:26.737209 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:26.737212 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:26.737299 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:26.737762 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:26.737764 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:26.737834 (+ 70us) client_negotiation.cc:769] Sending connection context
0808 08:25:26.737911 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 08:25:26.738035 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4150,"mutex_wait_us":359,"thread_start_us":97,"threads_started":1}
I20260808 08:25:26.738189 3278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 42359 } attrs { promote: false } } peers { permanent_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558" member_type: NON_VOTER last_known_addr { host: "127.22.235.252" port: 42345 } attrs { promote: true } } }
I20260808 08:25:26.738315 3265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.716115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:33539 (local address 127.22.235.254:46527)
0808 08:25:26.717875 (+ 1760us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:26.717880 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:26.720349 (+ 2469us) server_negotiation.cc:407] Connection header received
0808 08:25:26.722325 (+ 1976us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:26.722328 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:26.722377 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:26.722437 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:26.725760 (+ 3323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.726522 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.731658 (+ 5136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.731991 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.732065 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.734457 (+ 2392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:26.734461 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:26.734463 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:26.734488 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:26.736551 (+ 2063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:26.737017 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:26.737020 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:26.737022 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:26.737115 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:26.737406 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:26.737409 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:26.737411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:26.737628 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:26.737721 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:26.738011 (+ 290us) server_negotiation.cc:299] Negotiation successful
0808 08:25:26.738123 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1687,"thread_start_us":89,"threads_started":1}
I20260808 08:25:26.738339 3278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 [sys.catalog]: This master's current role is: LEADER
I20260808 08:25:26.738814 3244 heartbeater.cc:347] Connected to a master server at 127.22.235.254:46527
I20260808 08:25:26.738899 3244 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:26.739055 3244 heartbeater.cc:511] Master 127.22.235.254:46527 requested a full tablet report, sending...
I20260808 08:25:26.739177 2169 ts_manager.cc:194] Registered new tserver with Master: 27bb37319a6742ccb01e572b068b4bc9 (127.22.235.195:40217)
I20260808 08:25:26.739620 1158 ts_manager.cc:194] Re-registered known tserver with Master: 27bb37319a6742ccb01e572b068b4bc9 (127.22.235.195:40217)
I20260808 08:25:26.740263 1158 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.235.195@KRBTEST.COM'} at 127.22.235.195:33539
I20260808 08:25:26.742138 2095 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139: attempting to promote NON_VOTER b79f9c81ea0b4abeadd4ac1dbf54f558 to VOTER
I20260808 08:25:26.742465 2095 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 42359 } attrs { promote: false } } peers { permanent_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 42345 } attrs { promote: false } }
I20260808 08:25:26.743595 2494 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558 [term 1 LEARNER]: Refusing update from remote peer 00ae12b3a32549bb9d8a3092fa6e0139: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 08:25:26.743690 2189 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [term 1 FOLLOWER]: Refusing update from remote peer 00ae12b3a32549bb9d8a3092fa6e0139: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 08:25:26.744092 3279 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 42345 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260808 08:25:26.744365 3279 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 42359 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260808 08:25:26.745813 3278 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, b79f9c81ea0b4abeadd4ac1dbf54f558 (127.22.235.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 42359 } attrs { promote: false } } peers { permanent_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 42345 } attrs { promote: false } } }
I20260808 08:25:26.746389 2494 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, b79f9c81ea0b4abeadd4ac1dbf54f558 (127.22.235.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 42359 } attrs { promote: false } } peers { permanent_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 42345 } attrs { promote: false } } }
I20260808 08:25:26.747385 3278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 42359 } attrs { promote: false } } peers { permanent_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 42345 } attrs { promote: false } } }
I20260808 08:25:26.747463 3278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 [sys.catalog]: This master's current role is: LEADER
I20260808 08:25:26.747552 3278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 42359 } attrs { promote: false } } peers { permanent_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 42345 } attrs { promote: false } } }
I20260808 08:25:26.747617 3278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 [sys.catalog]: This master's current role is: LEADER
I20260808 08:25:26.747618 2095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 42359 } attrs { promote: false } } peers { permanent_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 42345 } attrs { promote: false } } }
I20260808 08:25:26.747732 2095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139 [sys.catalog]: This master's current role is: LEADER
I20260808 08:25:26.747784 2189 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, b79f9c81ea0b4abeadd4ac1dbf54f558 (127.22.235.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 42359 } attrs { promote: false } } peers { permanent_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 42345 } attrs { promote: false } } }
I20260808 08:25:26.748376 2547 sys_catalog.cc:455] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 42359 } attrs { promote: false } } peers { permanent_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 42345 } attrs { promote: false } } }
I20260808 08:25:26.748482 2547 sys_catalog.cc:458] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 08:25:26.754488 3296 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 42359 } attrs { promote: false } } peers { permanent_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 42345 } attrs { promote: false } } }
I20260808 08:25:26.754617 3296 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 08:25:26.756131 3044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.750698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.195:40217 (local address 127.0.0.1:54838)
0808 08:25:26.750868 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:26.750884 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:26.750964 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:26.751884 (+ 920us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:26.751887 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:26.751907 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:25:26.752002 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:26.752009 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.753486 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.753490 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:26.754520 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:26.754531 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.755730 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.755733 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:26.755829 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.755873 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 08:25:26.755934 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 08:25:26.756008 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 08:25:26.756371 3297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.750991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54838 (local address 127.22.235.195:40217)
0808 08:25:26.751348 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:26.751361 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:26.751386 (+ 25us) server_negotiation.cc:407] Connection header received
0808 08:25:26.751461 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:26.751478 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:26.751718 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:26.752167 (+ 449us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:25:26.752290 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.753349 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.754798 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.755587 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.755790 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.755928 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:26.755980 (+ 52us) server_negotiation.cc:299] Negotiation successful
0808 08:25:26.756064 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":146,"threads_started":1}
I20260808 08:25:26.756743 3298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.751237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.194:39211 (local address 127.0.0.1:40962)
0808 08:25:26.751464 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:26.751476 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:26.751570 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:26.752812 (+ 1242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:26.752815 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:26.752834 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:25:26.752935 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:26.752941 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.754341 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.754344 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:26.755359 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:26.755369 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.756353 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.756357 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:26.756458 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.756485 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 08:25:26.756529 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 08:25:26.756591 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":99,"threads_started":1}
I20260808 08:25:26.757272 3302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.751950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40962 (local address 127.22.235.194:39211)
0808 08:25:26.752338 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:26.752349 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:26.752370 (+ 21us) server_negotiation.cc:407] Connection header received
0808 08:25:26.752439 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:26.752443 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:26.752663 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:26.753072 (+ 409us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:25:26.753194 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.754208 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.755497 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.756212 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.756924 (+ 712us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.757035 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:26.757074 (+ 39us) server_negotiation.cc:299] Negotiation successful
0808 08:25:26.757137 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":157,"threads_started":1}
I20260808 08:25:26.757848 3299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.751558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:39011 (local address 127.0.0.1:58524)
0808 08:25:26.751765 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:26.751777 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:26.751856 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:26.752550 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:26.752553 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:26.752570 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:25:26.752700 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:26.752707 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.755082 (+ 2375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.755086 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:26.756261 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:26.756272 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.757465 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.757469 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:26.757567 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.757589 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 08:25:26.757670 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 08:25:26.757730 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":85,"threads_started":1}
I20260808 08:25:26.758044 3301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.751738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58524 (local address 127.22.235.193:39011)
0808 08:25:26.752104 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:26.752118 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:26.752135 (+ 17us) server_negotiation.cc:407] Connection header received
0808 08:25:26.752207 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:26.752212 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:26.752400 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:26.752855 (+ 455us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:25:26.752987 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.754987 (+ 2000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.756481 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.757356 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.757483 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.757600 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:26.757696 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 08:25:26.757782 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":109,"threads_started":1}
I20260808 08:25:26.759411 3121 tablet_service.cc:1467] Processing CreateTablet for tablet 8391fe5a501549f487439f7b403c05ec (DEFAULT_TABLE table=test-table [id=45536d90c5a847cb873d130f4cd94bbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:25:26.759928 2901 tablet_service.cc:1467] Processing CreateTablet for tablet 8391fe5a501549f487439f7b403c05ec (DEFAULT_TABLE table=test-table [id=45536d90c5a847cb873d130f4cd94bbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:25:26.760346 3121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8391fe5a501549f487439f7b403c05ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:26.761014 2901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8391fe5a501549f487439f7b403c05ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:26.760649 2667 tablet_service.cc:1467] Processing CreateTablet for tablet 8391fe5a501549f487439f7b403c05ec (DEFAULT_TABLE table=test-table [id=45536d90c5a847cb873d130f4cd94bbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:25:26.761631 2667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8391fe5a501549f487439f7b403c05ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:26.768025 3303 tablet_bootstrap.cc:492] T 8391fe5a501549f487439f7b403c05ec P 27bb37319a6742ccb01e572b068b4bc9: Bootstrap starting.
I20260808 08:25:26.769459 3304 tablet_bootstrap.cc:492] T 8391fe5a501549f487439f7b403c05ec P 11de37660d0946beb33ae725e791d907: Bootstrap starting.
I20260808 08:25:26.769459 3306 tablet_bootstrap.cc:492] T 8391fe5a501549f487439f7b403c05ec P 6b3a62b7e6a249d197410707a5bc428a: Bootstrap starting.
I20260808 08:25:26.770380 3303 tablet_bootstrap.cc:654] T 8391fe5a501549f487439f7b403c05ec P 27bb37319a6742ccb01e572b068b4bc9: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:26.771409 3303 log.cc:824] T 8391fe5a501549f487439f7b403c05ec P 27bb37319a6742ccb01e572b068b4bc9: Log is configured to *not* fsync() on all Append() calls
I20260808 08:25:26.771615 3306 tablet_bootstrap.cc:654] T 8391fe5a501549f487439f7b403c05ec P 6b3a62b7e6a249d197410707a5bc428a: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:26.771670 3304 tablet_bootstrap.cc:654] T 8391fe5a501549f487439f7b403c05ec P 11de37660d0946beb33ae725e791d907: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:26.772444 3304 log.cc:824] T 8391fe5a501549f487439f7b403c05ec P 11de37660d0946beb33ae725e791d907: Log is configured to *not* fsync() on all Append() calls
I20260808 08:25:26.772444 3306 log.cc:824] T 8391fe5a501549f487439f7b403c05ec P 6b3a62b7e6a249d197410707a5bc428a: Log is configured to *not* fsync() on all Append() calls
I20260808 08:25:26.773524 3303 tablet_bootstrap.cc:492] T 8391fe5a501549f487439f7b403c05ec P 27bb37319a6742ccb01e572b068b4bc9: No bootstrap required, opened a new log
I20260808 08:25:26.773710 3303 ts_tablet_manager.cc:1397] T 8391fe5a501549f487439f7b403c05ec P 27bb37319a6742ccb01e572b068b4bc9: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 08:25:26.774358 3304 tablet_bootstrap.cc:492] T 8391fe5a501549f487439f7b403c05ec P 11de37660d0946beb33ae725e791d907: No bootstrap required, opened a new log
I20260808 08:25:26.774358 3306 tablet_bootstrap.cc:492] T 8391fe5a501549f487439f7b403c05ec P 6b3a62b7e6a249d197410707a5bc428a: No bootstrap required, opened a new log
I20260808 08:25:26.774530 3306 ts_tablet_manager.cc:1397] T 8391fe5a501549f487439f7b403c05ec P 6b3a62b7e6a249d197410707a5bc428a: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260808 08:25:26.774531 3304 ts_tablet_manager.cc:1397] T 8391fe5a501549f487439f7b403c05ec P 11de37660d0946beb33ae725e791d907: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 08:25:26.778903 3306 raft_consensus.cc:348] T 8391fe5a501549f487439f7b403c05ec P 6b3a62b7e6a249d197410707a5bc428a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27bb37319a6742ccb01e572b068b4bc9" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 40217 } } peers { permanent_uuid: "6b3a62b7e6a249d197410707a5bc428a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39211 } } peers { permanent_uuid: "11de37660d0946beb33ae725e791d907" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 39011 } }
I20260808 08:25:26.779186 3306 raft_consensus.cc:374] T 8391fe5a501549f487439f7b403c05ec P 6b3a62b7e6a249d197410707a5bc428a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:26.779240 3306 raft_consensus.cc:729] T 8391fe5a501549f487439f7b403c05ec P 6b3a62b7e6a249d197410707a5bc428a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b3a62b7e6a249d197410707a5bc428a, State: Initialized, Role: FOLLOWER
I20260808 08:25:26.779165 3304 raft_consensus.cc:348] T 8391fe5a501549f487439f7b403c05ec P 11de37660d0946beb33ae725e791d907 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27bb37319a6742ccb01e572b068b4bc9" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 40217 } } peers { permanent_uuid: "6b3a62b7e6a249d197410707a5bc428a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39211 } } peers { permanent_uuid: "11de37660d0946beb33ae725e791d907" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 39011 } }
I20260808 08:25:26.779397 3304 raft_consensus.cc:374] T 8391fe5a501549f487439f7b403c05ec P 11de37660d0946beb33ae725e791d907 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:26.779222 3303 raft_consensus.cc:348] T 8391fe5a501549f487439f7b403c05ec P 27bb37319a6742ccb01e572b068b4bc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27bb37319a6742ccb01e572b068b4bc9" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 40217 } } peers { permanent_uuid: "6b3a62b7e6a249d197410707a5bc428a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39211 } } peers { permanent_uuid: "11de37660d0946beb33ae725e791d907" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 39011 } }
I20260808 08:25:26.779444 3304 raft_consensus.cc:729] T 8391fe5a501549f487439f7b403c05ec P 11de37660d0946beb33ae725e791d907 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11de37660d0946beb33ae725e791d907, State: Initialized, Role: FOLLOWER
I20260808 08:25:26.779465 3303 raft_consensus.cc:374] T 8391fe5a501549f487439f7b403c05ec P 27bb37319a6742ccb01e572b068b4bc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:26.779510 3303 raft_consensus.cc:729] T 8391fe5a501549f487439f7b403c05ec P 27bb37319a6742ccb01e572b068b4bc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27bb37319a6742ccb01e572b068b4bc9, State: Initialized, Role: FOLLOWER
I20260808 08:25:26.779745 3306 consensus_queue.cc:260] T 8391fe5a501549f487439f7b403c05ec P 6b3a62b7e6a249d197410707a5bc428a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27bb37319a6742ccb01e572b068b4bc9" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 40217 } } peers { permanent_uuid: "6b3a62b7e6a249d197410707a5bc428a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39211 } } peers { permanent_uuid: "11de37660d0946beb33ae725e791d907" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 39011 } }
I20260808 08:25:26.779868 3304 consensus_queue.cc:260] T 8391fe5a501549f487439f7b403c05ec P 11de37660d0946beb33ae725e791d907 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27bb37319a6742ccb01e572b068b4bc9" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 40217 } } peers { permanent_uuid: "6b3a62b7e6a249d197410707a5bc428a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39211 } } peers { permanent_uuid: "11de37660d0946beb33ae725e791d907" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 39011 } }
I20260808 08:25:26.779986 3303 consensus_queue.cc:260] T 8391fe5a501549f487439f7b403c05ec P 27bb37319a6742ccb01e572b068b4bc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27bb37319a6742ccb01e572b068b4bc9" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 40217 } } peers { permanent_uuid: "6b3a62b7e6a249d197410707a5bc428a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39211 } } peers { permanent_uuid: "11de37660d0946beb33ae725e791d907" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 39011 } }
I20260808 08:25:26.780799 3304 ts_tablet_manager.cc:1428] T 8391fe5a501549f487439f7b403c05ec P 11de37660d0946beb33ae725e791d907: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 08:25:26.780799 3306 ts_tablet_manager.cc:1428] T 8391fe5a501549f487439f7b403c05ec P 6b3a62b7e6a249d197410707a5bc428a: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 08:25:26.781038 3303 ts_tablet_manager.cc:1428] T 8391fe5a501549f487439f7b403c05ec P 27bb37319a6742ccb01e572b068b4bc9: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260808 08:25:26.782941 3244 heartbeater.cc:503] Master 127.22.235.254:46527 was elected leader, sending a full tablet report...
I20260808 08:25:26.783252 3007 heartbeater.cc:503] Master 127.22.235.254:46527 was elected leader, sending a full tablet report...
I20260808 08:25:26.785069 2791 heartbeater.cc:503] Master 127.22.235.254:46527 was elected leader, sending a full tablet report...
I20260808 08:25:26.785730 3318 raft_consensus.cc:482] T 8391fe5a501549f487439f7b403c05ec P 6b3a62b7e6a249d197410707a5bc428a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 08:25:26.785876 3318 raft_consensus.cc:504] T 8391fe5a501549f487439f7b403c05ec P 6b3a62b7e6a249d197410707a5bc428a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27bb37319a6742ccb01e572b068b4bc9" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 40217 } } peers { permanent_uuid: "6b3a62b7e6a249d197410707a5bc428a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39211 } } peers { permanent_uuid: "11de37660d0946beb33ae725e791d907" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 39011 } }
I20260808 08:25:26.791054 3318 leader_election.cc:288] T 8391fe5a501549f487439f7b403c05ec P 6b3a62b7e6a249d197410707a5bc428a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27bb37319a6742ccb01e572b068b4bc9 (127.22.235.195:40217), 11de37660d0946beb33ae725e791d907 (127.22.235.193:39011)
I20260808 08:25:26.795852 3012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.791206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.195:40217 (local address 127.22.235.194:51879)
0808 08:25:26.791365 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:26.791380 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:26.791462 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:26.791896 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:26.791900 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:26.791927 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:25:26.792111 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:26.792119 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.793174 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.793178 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:26.794571 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:26.794582 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.795420 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.795424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:26.795552 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.795579 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 08:25:26.795653 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 08:25:26.795748 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"mutex_wait_us":96,"spinlock_wait_cycles":7296}
I20260808 08:25:26.795872 3297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.791456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:51879 (local address 127.22.235.195:40217)
0808 08:25:26.791617 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:26.791621 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:26.791633 (+ 12us) server_negotiation.cc:407] Connection header received
0808 08:25:26.791676 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:26.791679 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:26.791721 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:26.791784 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:25:26.792232 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.793079 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.794715 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.795311 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.795431 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.795484 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:26.795700 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 08:25:26.795780 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260808 08:25:26.796314 3015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.791206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:39011 (local address 127.22.235.194:54439)
0808 08:25:26.791520 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:26.791534 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:26.791610 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:26.791896 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:26.791900 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:26.791929 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:25:26.792117 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:26.792123 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.793215 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.793218 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:26.794571 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:26.794582 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.795420 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.795424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:26.796036 (+ 612us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.796061 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 08:25:26.796128 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 08:25:26.796198 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":658,"spinlock_wait_cycles":6656}
I20260808 08:25:26.796329 3301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.791521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:54439 (local address 127.22.235.193:39011)
0808 08:25:26.791634 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:26.791638 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:26.791665 (+ 27us) server_negotiation.cc:407] Connection header received
0808 08:25:26.791699 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:26.791702 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:26.791739 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:26.791796 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:25:26.792232 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.793132 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.794715 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.795318 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.795431 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.795484 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:26.796157 (+ 673us) server_negotiation.cc:299] Negotiation successful
0808 08:25:26.796228 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 08:25:26.796425 3161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8391fe5a501549f487439f7b403c05ec" candidate_uuid: "6b3a62b7e6a249d197410707a5bc428a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27bb37319a6742ccb01e572b068b4bc9" is_pre_election: true
I20260808 08:25:26.796731 3161 raft_consensus.cc:2454] T 8391fe5a501549f487439f7b403c05ec P 27bb37319a6742ccb01e572b068b4bc9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b3a62b7e6a249d197410707a5bc428a in term 0.
I20260808 08:25:26.796864 2709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8391fe5a501549f487439f7b403c05ec" candidate_uuid: "6b3a62b7e6a249d197410707a5bc428a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11de37660d0946beb33ae725e791d907" is_pre_election: true
I20260808 08:25:26.797114 2709 raft_consensus.cc:2454] T 8391fe5a501549f487439f7b403c05ec P 11de37660d0946beb33ae725e791d907 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b3a62b7e6a249d197410707a5bc428a in term 0.
I20260808 08:25:26.797201 2833 leader_election.cc:302] T 8391fe5a501549f487439f7b403c05ec P 6b3a62b7e6a249d197410707a5bc428a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27bb37319a6742ccb01e572b068b4bc9, 6b3a62b7e6a249d197410707a5bc428a; no voters:
I20260808 08:25:26.797670 3318 raft_consensus.cc:2781] T 8391fe5a501549f487439f7b403c05ec P 6b3a62b7e6a249d197410707a5bc428a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 08:25:26.797744 3318 raft_consensus.cc:482] T 8391fe5a501549f487439f7b403c05ec P 6b3a62b7e6a249d197410707a5bc428a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 08:25:26.797786 3318 raft_consensus.cc:3037] T 8391fe5a501549f487439f7b403c05ec P 6b3a62b7e6a249d197410707a5bc428a [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:26.799072 3318 raft_consensus.cc:504] T 8391fe5a501549f487439f7b403c05ec P 6b3a62b7e6a249d197410707a5bc428a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27bb37319a6742ccb01e572b068b4bc9" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 40217 } } peers { permanent_uuid: "6b3a62b7e6a249d197410707a5bc428a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39211 } } peers { permanent_uuid: "11de37660d0946beb33ae725e791d907" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 39011 } }
I20260808 08:25:26.799391 3318 leader_election.cc:288] T 8391fe5a501549f487439f7b403c05ec P 6b3a62b7e6a249d197410707a5bc428a [CANDIDATE]: Term 1 election: Requested vote from peers 27bb37319a6742ccb01e572b068b4bc9 (127.22.235.195:40217), 11de37660d0946beb33ae725e791d907 (127.22.235.193:39011)
I20260808 08:25:26.799703 2709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8391fe5a501549f487439f7b403c05ec" candidate_uuid: "6b3a62b7e6a249d197410707a5bc428a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11de37660d0946beb33ae725e791d907"
I20260808 08:25:26.799826 2709 raft_consensus.cc:3037] T 8391fe5a501549f487439f7b403c05ec P 11de37660d0946beb33ae725e791d907 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:26.800724 3161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8391fe5a501549f487439f7b403c05ec" candidate_uuid: "6b3a62b7e6a249d197410707a5bc428a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27bb37319a6742ccb01e572b068b4bc9"
I20260808 08:25:26.800827 3161 raft_consensus.cc:3037] T 8391fe5a501549f487439f7b403c05ec P 27bb37319a6742ccb01e572b068b4bc9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:26.800945 2709 raft_consensus.cc:2454] T 8391fe5a501549f487439f7b403c05ec P 11de37660d0946beb33ae725e791d907 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b3a62b7e6a249d197410707a5bc428a in term 1.
I20260808 08:25:26.801245 2828 leader_election.cc:302] T 8391fe5a501549f487439f7b403c05ec P 6b3a62b7e6a249d197410707a5bc428a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11de37660d0946beb33ae725e791d907, 6b3a62b7e6a249d197410707a5bc428a; no voters:
I20260808 08:25:26.801426 3318 raft_consensus.cc:2781] T 8391fe5a501549f487439f7b403c05ec P 6b3a62b7e6a249d197410707a5bc428a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:25:26.801508 3318 raft_consensus.cc:686] T 8391fe5a501549f487439f7b403c05ec P 6b3a62b7e6a249d197410707a5bc428a [term 1 LEADER]: Becoming Leader. State: Replica: 6b3a62b7e6a249d197410707a5bc428a, State: Running, Role: LEADER
I20260808 08:25:26.801846 3318 consensus_queue.cc:237] T 8391fe5a501549f487439f7b403c05ec P 6b3a62b7e6a249d197410707a5bc428a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27bb37319a6742ccb01e572b068b4bc9" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 40217 } } peers { permanent_uuid: "6b3a62b7e6a249d197410707a5bc428a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39211 } } peers { permanent_uuid: "11de37660d0946beb33ae725e791d907" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 39011 } }
I20260808 08:25:26.801955 3161 raft_consensus.cc:2454] T 8391fe5a501549f487439f7b403c05ec P 27bb37319a6742ccb01e572b068b4bc9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b3a62b7e6a249d197410707a5bc428a in term 1.
I20260808 08:25:26.805481 1155 catalog_manager.cc:5495] T 8391fe5a501549f487439f7b403c05ec P 6b3a62b7e6a249d197410707a5bc428a reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b3a62b7e6a249d197410707a5bc428a (127.22.235.194). New cstate: current_term: 1 leader_uuid: "6b3a62b7e6a249d197410707a5bc428a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27bb37319a6742ccb01e572b068b4bc9" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 40217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b3a62b7e6a249d197410707a5bc428a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39211 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11de37660d0946beb33ae725e791d907" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 39011 } health_report { overall_health: UNKNOWN } } }
I20260808 08:25:26.819269 3302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.815787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40978 (local address 127.22.235.194:39211)
0808 08:25:26.815917 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:26.815922 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:26.815935 (+ 13us) server_negotiation.cc:407] Connection header received
0808 08:25:26.816071 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:26.816074 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:26.816128 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:26.816213 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:25:26.816473 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.817023 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.817855 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.818068 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.818152 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.818447 (+ 295us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:25:26.818864 (+ 417us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 08:25:26.818968 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:26.819061 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 08:25:26.819115 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 08:25:26.824883 2709 raft_consensus.cc:1261] T 8391fe5a501549f487439f7b403c05ec P 11de37660d0946beb33ae725e791d907 [term 1 FOLLOWER]: Refusing update from remote peer 6b3a62b7e6a249d197410707a5bc428a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:25:26.824883 3161 raft_consensus.cc:1261] T 8391fe5a501549f487439f7b403c05ec P 27bb37319a6742ccb01e572b068b4bc9 [term 1 FOLLOWER]: Refusing update from remote peer 6b3a62b7e6a249d197410707a5bc428a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:25:26.825582 3319 consensus_queue.cc:1035] T 8391fe5a501549f487439f7b403c05ec P 6b3a62b7e6a249d197410707a5bc428a [LEADER]: Connected to new peer: Peer: permanent_uuid: "11de37660d0946beb33ae725e791d907" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 39011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:25:26.825755 3318 consensus_queue.cc:1035] T 8391fe5a501549f487439f7b403c05ec P 6b3a62b7e6a249d197410707a5bc428a [LEADER]: Connected to new peer: Peer: permanent_uuid: "27bb37319a6742ccb01e572b068b4bc9" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 40217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:25:26.834312 3329 mvcc.cc:205] Tried to move back new op lower bound from 7316183149867655168 to 7316183149783724032. Current Snapshot: MvccSnapshot[applied={T|T < 7316183149867655168}]
I20260808 08:25:26.834568 3330 mvcc.cc:205] Tried to move back new op lower bound from 7316183149867655168 to 7316183149783724032. Current Snapshot: MvccSnapshot[applied={T|T < 7316183149867655168}]
I20260808 08:25:26.846768 1155 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44756:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:25:26.846941 1155 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44756:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:25:26.851174 1155 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 00ae12b3a32549bb9d8a3092fa6e0139: Sending DeleteTablet for 3 replicas of tablet 8391fe5a501549f487439f7b403c05ec
I20260808 08:25:26.851940 3121 tablet_service.cc:1514] Processing DeleteTablet for tablet 8391fe5a501549f487439f7b403c05ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:25:26 UTC) from {username='oryx', principal='oryx/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:54838
I20260808 08:25:26.852171 2901 tablet_service.cc:1514] Processing DeleteTablet for tablet 8391fe5a501549f487439f7b403c05ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:25:26 UTC) from {username='oryx', principal='oryx/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:40962
I20260808 08:25:26.852730 2667 tablet_service.cc:1514] Processing DeleteTablet for tablet 8391fe5a501549f487439f7b403c05ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:25:26 UTC) from {username='oryx', principal='oryx/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:58524
I20260808 08:25:26.853035 3349 tablet_replica.cc:330] T 8391fe5a501549f487439f7b403c05ec P 6b3a62b7e6a249d197410707a5bc428a: stopping tablet replica
I20260808 08:25:26.853653 3349 raft_consensus.cc:2229] T 8391fe5a501549f487439f7b403c05ec P 6b3a62b7e6a249d197410707a5bc428a [term 1 LEADER]: Raft consensus shutting down.
I20260808 08:25:26.853850 3348 tablet_replica.cc:330] T 8391fe5a501549f487439f7b403c05ec P 27bb37319a6742ccb01e572b068b4bc9: stopping tablet replica
I20260808 08:25:26.854120 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 1099
I20260808 08:25:26.854272 3349 raft_consensus.cc:2258] T 8391fe5a501549f487439f7b403c05ec P 6b3a62b7e6a249d197410707a5bc428a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:25:26.865547 3349 ts_tablet_manager.cc:1905] T 8391fe5a501549f487439f7b403c05ec P 6b3a62b7e6a249d197410707a5bc428a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:25:26.854146 3348 raft_consensus.cc:2229] T 8391fe5a501549f487439f7b403c05ec P 27bb37319a6742ccb01e572b068b4bc9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 08:25:26.866415 3348 raft_consensus.cc:2258] T 8391fe5a501549f487439f7b403c05ec P 27bb37319a6742ccb01e572b068b4bc9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:25:26.868191 3348 ts_tablet_manager.cc:1905] T 8391fe5a501549f487439f7b403c05ec P 27bb37319a6742ccb01e572b068b4bc9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:25:26.868883 3349 ts_tablet_manager.cc:1918] T 8391fe5a501549f487439f7b403c05ec P 6b3a62b7e6a249d197410707a5bc428a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 08:25:26.869048 3349 log.cc:1196] T 8391fe5a501549f487439f7b403c05ec P 6b3a62b7e6a249d197410707a5bc428a: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-1/wal/wals/8391fe5a501549f487439f7b403c05ec
I20260808 08:25:26.869453 3349 ts_tablet_manager.cc:1939] T 8391fe5a501549f487439f7b403c05ec P 6b3a62b7e6a249d197410707a5bc428a: Deleting consensus metadata
I20260808 08:25:26.871011 3348 ts_tablet_manager.cc:1918] T 8391fe5a501549f487439f7b403c05ec P 27bb37319a6742ccb01e572b068b4bc9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 08:25:26.871111 3348 log.cc:1196] T 8391fe5a501549f487439f7b403c05ec P 27bb37319a6742ccb01e572b068b4bc9: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-2/wal/wals/8391fe5a501549f487439f7b403c05ec
I20260808 08:25:26.871284 3351 tablet_replica.cc:330] T 8391fe5a501549f487439f7b403c05ec P 11de37660d0946beb33ae725e791d907: stopping tablet replica
I20260808 08:25:26.871419 3348 ts_tablet_manager.cc:1939] T 8391fe5a501549f487439f7b403c05ec P 27bb37319a6742ccb01e572b068b4bc9: Deleting consensus metadata
I20260808 08:25:26.871721 3351 raft_consensus.cc:2229] T 8391fe5a501549f487439f7b403c05ec P 11de37660d0946beb33ae725e791d907 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 08:25:26.872102 3351 raft_consensus.cc:2258] T 8391fe5a501549f487439f7b403c05ec P 11de37660d0946beb33ae725e791d907 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:25:26.873507 3351 ts_tablet_manager.cc:1905] T 8391fe5a501549f487439f7b403c05ec P 11de37660d0946beb33ae725e791d907: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 08:25:26.875037 2830 connection.cc:535] server connection from 127.0.0.1:40962 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 08:25:26.875447 3047 connection.cc:535] server connection from 127.0.0.1:54838 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 08:25:26.875660 3351 ts_tablet_manager.cc:1918] T 8391fe5a501549f487439f7b403c05ec P 11de37660d0946beb33ae725e791d907: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 08:25:26.875742 3351 log.cc:1196] T 8391fe5a501549f487439f7b403c05ec P 11de37660d0946beb33ae725e791d907: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-0/wal/wals/8391fe5a501549f487439f7b403c05ec
I20260808 08:25:26.875981 3351 ts_tablet_manager.cc:1939] T 8391fe5a501549f487439f7b403c05ec P 11de37660d0946beb33ae725e791d907: Deleting consensus metadata
W20260808 08:25:26.877050 2596 connection.cc:441] server connection from 127.0.0.1:58524 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:58524 (request call id 1) could send its response
I20260808 08:25:26.886411 3272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.853376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51862 (local address 127.22.235.253:42359)
0808 08:25:26.853486 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:26.853491 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:26.876187 (+ 22696us) server_negotiation.cc:407] Connection header received
0808 08:25:26.876436 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:26.876439 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:26.876512 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:26.876598 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:26.877788 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.878342 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.879530 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.879899 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.879975 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.881859 (+ 1884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:26.881863 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:26.881866 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:26.881893 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:26.883517 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:26.884185 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:26.884189 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:26.884190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:26.884274 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:26.884571 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:26.884575 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:26.884577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:26.884810 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:26.884936 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:26.886134 (+ 1198us) server_negotiation.cc:299] Negotiation successful
0808 08:25:26.886248 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 08:25:26.889726 2804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.879428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46434 (local address 127.22.235.252:42345)
0808 08:25:26.879543 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:26.879547 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:26.880686 (+ 1139us) server_negotiation.cc:407] Connection header received
0808 08:25:26.880894 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:26.880897 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:26.880947 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:26.880997 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:26.882203 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.882991 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.884185 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.884533 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.884604 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.886392 (+ 1788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:26.886395 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:26.886398 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:26.886423 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:26.888152 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:26.888536 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:26.888540 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:26.888541 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:26.888663 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:26.888903 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:26.888906 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:26.888908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:26.889078 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:26.889193 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:26.889458 (+ 265us) server_negotiation.cc:299] Negotiation successful
0808 08:25:26.889577 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 08:25:26.893674 3273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.853677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46428 (local address 127.22.235.252:42345)
0808 08:25:26.853784 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:26.853788 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:26.885936 (+ 32148us) server_negotiation.cc:407] Connection header received
0808 08:25:26.886053 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:26.886056 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:26.886107 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:26.886185 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:26.887694 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.888218 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.889274 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.889576 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.889673 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.891045 (+ 1372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:26.891048 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:26.891050 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:26.891069 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:26.892202 (+ 1133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:26.892650 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:26.892653 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:26.892655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:26.892724 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:26.892956 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:26.892958 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:26.892960 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:26.893112 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:26.893204 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:26.893455 (+ 251us) server_negotiation.cc:299] Negotiation successful
0808 08:25:26.893545 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 08:25:26.898032 2376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:26.879367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51868 (local address 127.22.235.253:42359)
0808 08:25:26.879480 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:26.879484 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:26.879496 (+ 12us) server_negotiation.cc:407] Connection header received
0808 08:25:26.879530 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:26.879533 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:26.879575 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:26.879646 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:26.881154 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.881898 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.883012 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:26.883347 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:26.883425 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:26.885039 (+ 1614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:26.885042 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:26.885045 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:26.885067 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:26.886592 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:26.896953 (+ 10361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:26.896956 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:26.896958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:26.897041 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:26.897284 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:26.897288 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:26.897290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:26.897455 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:26.897575 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:26.897783 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 08:25:26.897888 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 08:25:27.838620 3365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:27.838151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46527 (local address 127.22.235.195:43387)
0808 08:25:27.838389 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:27.838522 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":74,"threads_started":1}
W20260808 08:25:27.839097 3244 heartbeater.cc:644] Failed to heartbeat to 127.22.235.254:46527 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.22.235.254:46527: connect: Connection refused (error 111)
I20260808 08:25:27.840349 3366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:27.839968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46527 (local address 127.22.235.193:53777)
0808 08:25:27.840191 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:27.840271 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":64,"threads_started":1}
W20260808 08:25:27.840631 2791 heartbeater.cc:644] Failed to heartbeat to 127.22.235.254:46527 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.22.235.254:46527: connect: Connection refused (error 111)
I20260808 08:25:27.841522 3367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:27.841115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46527 (local address 127.22.235.194:59645)
0808 08:25:27.841348 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:27.841435 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":63,"threads_started":1}
W20260808 08:25:27.841735 3007 heartbeater.cc:644] Failed to heartbeat to 127.22.235.254:46527 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.22.235.254:46527: connect: Connection refused (error 111)
I20260808 08:25:28.351614 3368 raft_consensus.cc:482] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 00ae12b3a32549bb9d8a3092fa6e0139)
I20260808 08:25:28.351614 3369 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 00ae12b3a32549bb9d8a3092fa6e0139)
I20260808 08:25:28.351828 3368 raft_consensus.cc:504] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 42359 } attrs { promote: false } } peers { permanent_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 42345 } attrs { promote: false } }
I20260808 08:25:28.351831 3369 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 42359 } attrs { promote: false } } peers { permanent_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 42345 } attrs { promote: false } }
I20260808 08:25:28.352873 3368 leader_election.cc:288] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 00ae12b3a32549bb9d8a3092fa6e0139 (127.22.235.254:46527), ad643f833e2a4adc89d9c6c143dbb971 (127.22.235.253:42359)
I20260808 08:25:28.352877 3369 leader_election.cc:288] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 00ae12b3a32549bb9d8a3092fa6e0139 (127.22.235.254:46527), b79f9c81ea0b4abeadd4ac1dbf54f558 (127.22.235.252:42345)
I20260808 08:25:28.353847 3374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:28.353098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46527 (local address 127.0.0.1:45202)
0808 08:25:28.353644 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:28.353732 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":53,"threads_started":1}
I20260808 08:25:28.353907 3370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:28.353047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46527 (local address 127.0.0.1:45214)
0808 08:25:28.353729 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:28.353808 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":55,"threads_started":1}
W20260808 08:25:28.354084 2445 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46527: connect: Connection refused (error 111)
W20260808 08:25:28.354084 2141 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46527: connect: Connection refused (error 111)
I20260808 08:25:28.355048 3374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:28.354725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46527 (local address 127.0.0.1:45224)
0808 08:25:28.354900 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:28.354953 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":34}
I20260808 08:25:28.355310 3370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:28.355060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46527 (local address 127.0.0.1:45234)
0808 08:25:28.355197 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:28.355244 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260808 08:25:28.355279 2445 leader_election.cc:334] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 00ae12b3a32549bb9d8a3092fa6e0139 (127.22.235.254:46527): Network error: Client connection negotiation failed: client connection to 127.22.235.254:46527: connect: Connection refused (error 111)
W20260808 08:25:28.355407 2141 leader_election.cc:334] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 00ae12b3a32549bb9d8a3092fa6e0139 (127.22.235.254:46527): Network error: Client connection negotiation failed: client connection to 127.22.235.254:46527: connect: Connection refused (error 111)
I20260808 08:25:28.358251 3381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:28.353567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46438 (local address 127.22.235.252:42345)
0808 08:25:28.353741 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:28.353745 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:28.354013 (+ 268us) server_negotiation.cc:407] Connection header received
0808 08:25:28.354069 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:28.354073 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:28.354124 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:28.354201 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:25:28.354823 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:28.355597 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:28.357083 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:28.357660 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:28.357793 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:28.357849 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:28.358091 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 08:25:28.358150 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":50,"threads_started":1}
I20260808 08:25:28.358289 3376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:28.353339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:42345 (local address 127.0.0.1:46438)
0808 08:25:28.353780 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:28.353792 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:28.353909 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:28.354562 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:28.354565 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:28.354584 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:25:28.354694 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:28.354701 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:28.355745 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:28.355748 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:28.356960 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:28.356972 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:28.357796 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:28.357799 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:28.357914 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:28.357994 (+ 80us) client_negotiation.cc:769] Sending connection context
0808 08:25:28.358076 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 08:25:28.358151 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"mutex_wait_us":48,"thread_start_us":72,"threads_started":1}
I20260808 08:25:28.358769 2494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ad643f833e2a4adc89d9c6c143dbb971" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558" is_pre_election: true
I20260808 08:25:28.359002 3378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:28.353378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.253:42359 (local address 127.0.0.1:51878)
0808 08:25:28.354428 (+ 1050us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:28.354441 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:28.354550 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:28.355034 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:28.355037 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:28.355057 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:25:28.355152 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:28.355159 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:28.356435 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:28.356438 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:28.357682 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:28.357691 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:28.358637 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:28.358641 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:28.358729 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:28.358747 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 08:25:28.358809 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 08:25:28.358869 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":940,"mutex_wait_us":1,"thread_start_us":88,"threads_started":1}
I20260808 08:25:28.359009 2494 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad643f833e2a4adc89d9c6c143dbb971 in term 1.
I20260808 08:25:28.359225 3380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:28.353486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51878 (local address 127.22.235.253:42359)
0808 08:25:28.353884 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:28.353888 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:28.354517 (+ 629us) server_negotiation.cc:407] Connection header received
0808 08:25:28.354689 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:28.354693 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:28.354745 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:28.354922 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:25:28.355337 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:28.356153 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:28.357794 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:28.358429 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:28.358532 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:28.358581 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:28.358858 (+ 277us) server_negotiation.cc:299] Negotiation successful
0808 08:25:28.358947 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":334,"thread_start_us":77,"threads_started":1}
I20260808 08:25:28.359474 2141 leader_election.cc:302] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad643f833e2a4adc89d9c6c143dbb971, b79f9c81ea0b4abeadd4ac1dbf54f558; no voters: 00ae12b3a32549bb9d8a3092fa6e0139
I20260808 08:25:28.359616 2189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ad643f833e2a4adc89d9c6c143dbb971" is_pre_election: true
I20260808 08:25:28.359725 3369 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 08:25:28.359813 2189 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b79f9c81ea0b4abeadd4ac1dbf54f558 in term 1.
I20260808 08:25:28.359920 3369 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 00ae12b3a32549bb9d8a3092fa6e0139)
I20260808 08:25:28.359983 3369 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [term 1 FOLLOWER]: Advancing to term 2
I20260808 08:25:28.360188 2445 leader_election.cc:302] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad643f833e2a4adc89d9c6c143dbb971, b79f9c81ea0b4abeadd4ac1dbf54f558; no voters: 00ae12b3a32549bb9d8a3092fa6e0139
I20260808 08:25:28.360414 3368 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 08:25:28.360476 3368 raft_consensus.cc:482] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 00ae12b3a32549bb9d8a3092fa6e0139)
I20260808 08:25:28.360513 3368 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558 [term 1 FOLLOWER]: Advancing to term 2
I20260808 08:25:28.361351 3369 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 42359 } attrs { promote: false } } peers { permanent_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 42345 } attrs { promote: false } }
I20260808 08:25:28.361811 3369 leader_election.cc:288] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [CANDIDATE]: Term 2 election: Requested vote from peers 00ae12b3a32549bb9d8a3092fa6e0139 (127.22.235.254:46527), b79f9c81ea0b4abeadd4ac1dbf54f558 (127.22.235.252:42345)
I20260808 08:25:28.361780 3368 raft_consensus.cc:504] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 42359 } attrs { promote: false } } peers { permanent_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 42345 } attrs { promote: false } }
I20260808 08:25:28.362144 3368 leader_election.cc:288] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558 [CANDIDATE]: Term 2 election: Requested vote from peers 00ae12b3a32549bb9d8a3092fa6e0139 (127.22.235.254:46527), ad643f833e2a4adc89d9c6c143dbb971 (127.22.235.253:42359)
I20260808 08:25:28.362221 3376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:28.361970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46527 (local address 127.0.0.1:45246)
0808 08:25:28.362093 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:28.362152 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 08:25:28.362360 2494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ad643f833e2a4adc89d9c6c143dbb971" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558"
I20260808 08:25:28.362519 2494 raft_consensus.cc:2379] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ad643f833e2a4adc89d9c6c143dbb971 in current term 2: Already voted for candidate b79f9c81ea0b4abeadd4ac1dbf54f558 in this term.
I20260808 08:25:28.362636 2189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ad643f833e2a4adc89d9c6c143dbb971"
I20260808 08:25:28.362802 2189 raft_consensus.cc:2379] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b79f9c81ea0b4abeadd4ac1dbf54f558 in current term 2: Already voted for candidate ad643f833e2a4adc89d9c6c143dbb971 in this term.
I20260808 08:25:28.362841 3378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:28.362252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46527 (local address 127.0.0.1:45258)
0808 08:25:28.362725 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:28.362778 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410}
I20260808 08:25:28.362905 3376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:28.362688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46527 (local address 127.0.0.1:45272)
0808 08:25:28.362803 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:28.362850 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 08:25:28.362986 2141 leader_election.cc:334] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 00ae12b3a32549bb9d8a3092fa6e0139 (127.22.235.254:46527): Network error: Client connection negotiation failed: client connection to 127.22.235.254:46527: connect: Connection refused (error 111)
I20260808 08:25:28.363051 2141 leader_election.cc:302] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad643f833e2a4adc89d9c6c143dbb971; no voters: 00ae12b3a32549bb9d8a3092fa6e0139, b79f9c81ea0b4abeadd4ac1dbf54f558
I20260808 08:25:28.363204 3369 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260808 08:25:28.363745 3378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:28.363552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46527 (local address 127.0.0.1:45286)
0808 08:25:28.363652 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:28.363696 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 08:25:28.363838 2445 leader_election.cc:334] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 00ae12b3a32549bb9d8a3092fa6e0139 (127.22.235.254:46527): Network error: Client connection negotiation failed: client connection to 127.22.235.254:46527: connect: Connection refused (error 111)
I20260808 08:25:28.363909 2445 leader_election.cc:302] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b79f9c81ea0b4abeadd4ac1dbf54f558; no voters: 00ae12b3a32549bb9d8a3092fa6e0139, ad643f833e2a4adc89d9c6c143dbb971
I20260808 08:25:28.364063 3368 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260808 08:25:28.840338 3385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:28.839830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46527 (local address 127.22.235.195:57949)
0808 08:25:28.840141 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:28.840217 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":98,"threads_started":1}
I20260808 08:25:28.841504 3387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:28.841110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46527 (local address 127.22.235.193:59997)
0808 08:25:28.841358 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:28.841414 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":87,"threads_started":1}
I20260808 08:25:28.842715 3388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:28.842241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46527 (local address 127.22.235.194:56677)
0808 08:25:28.842472 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:28.842619 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":75,"threads_started":1}
I20260808 08:25:29.841979 3391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:29.841529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46527 (local address 127.22.235.195:33251)
0808 08:25:29.841788 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:29.841862 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":79,"threads_started":1}
I20260808 08:25:29.842633 3392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:29.842146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46527 (local address 127.22.235.193:59343)
0808 08:25:29.842474 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:29.842538 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":69,"threads_started":1}
I20260808 08:25:29.843787 3394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:29.843379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46527 (local address 127.22.235.194:50425)
0808 08:25:29.843656 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:29.843712 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":93,"threads_started":1}
I20260808 08:25:30.239548 3395 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 08:25:30.239707 3395 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 42359 } attrs { promote: false } } peers { permanent_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 42345 } attrs { promote: false } }
I20260808 08:25:30.240221 3395 leader_election.cc:288] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 00ae12b3a32549bb9d8a3092fa6e0139 (127.22.235.254:46527), b79f9c81ea0b4abeadd4ac1dbf54f558 (127.22.235.252:42345)
I20260808 08:25:30.240882 3397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:30.240419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46527 (local address 127.0.0.1:45542)
0808 08:25:30.240718 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:30.240788 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":108,"threads_started":1}
I20260808 08:25:30.241097 2494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ad643f833e2a4adc89d9c6c143dbb971" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558" is_pre_election: true
I20260808 08:25:30.241237 2494 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad643f833e2a4adc89d9c6c143dbb971 in term 2.
I20260808 08:25:30.241595 2141 leader_election.cc:302] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad643f833e2a4adc89d9c6c143dbb971, b79f9c81ea0b4abeadd4ac1dbf54f558; no voters:
I20260808 08:25:30.241775 3395 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260808 08:25:30.241844 3395 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 08:25:30.241884 3395 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [term 2 FOLLOWER]: Advancing to term 3
I20260808 08:25:30.242130 3397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:30.241891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46527 (local address 127.0.0.1:45556)
0808 08:25:30.242022 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:30.242072 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 08:25:30.242230 2141 leader_election.cc:334] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 00ae12b3a32549bb9d8a3092fa6e0139 (127.22.235.254:46527): Network error: Client connection negotiation failed: client connection to 127.22.235.254:46527: connect: Connection refused (error 111)
I20260808 08:25:30.243053 3395 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 42359 } attrs { promote: false } } peers { permanent_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 42345 } attrs { promote: false } }
I20260808 08:25:30.243355 3395 leader_election.cc:288] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [CANDIDATE]: Term 3 election: Requested vote from peers 00ae12b3a32549bb9d8a3092fa6e0139 (127.22.235.254:46527), b79f9c81ea0b4abeadd4ac1dbf54f558 (127.22.235.252:42345)
I20260808 08:25:30.243686 3397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:30.243465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46527 (local address 127.0.0.1:45572)
0808 08:25:30.243566 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:30.243618 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 08:25:30.243885 2494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ad643f833e2a4adc89d9c6c143dbb971" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558"
I20260808 08:25:30.244024 2494 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558 [term 2 FOLLOWER]: Advancing to term 3
I20260808 08:25:30.244336 3397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:30.244139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46527 (local address 127.0.0.1:45578)
0808 08:25:30.244236 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:30.244285 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 08:25:30.244459 2141 leader_election.cc:334] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 00ae12b3a32549bb9d8a3092fa6e0139 (127.22.235.254:46527): Network error: Client connection negotiation failed: client connection to 127.22.235.254:46527: connect: Connection refused (error 111)
I20260808 08:25:30.245117 2494 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad643f833e2a4adc89d9c6c143dbb971 in term 3.
I20260808 08:25:30.245407 2141 leader_election.cc:302] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad643f833e2a4adc89d9c6c143dbb971, b79f9c81ea0b4abeadd4ac1dbf54f558; no voters: 00ae12b3a32549bb9d8a3092fa6e0139
I20260808 08:25:30.245580 3395 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 08:25:30.245748 3395 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [term 3 LEADER]: Becoming Leader. State: Replica: ad643f833e2a4adc89d9c6c143dbb971, State: Running, Role: LEADER
I20260808 08:25:30.246045 3395 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 42359 } attrs { promote: false } } peers { permanent_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 42345 } attrs { promote: false } }
I20260808 08:25:30.247013 3399 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad643f833e2a4adc89d9c6c143dbb971. Latest consensus state: current_term: 3 leader_uuid: "ad643f833e2a4adc89d9c6c143dbb971" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 42359 } attrs { promote: false } } peers { permanent_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 42345 } attrs { promote: false } } }
I20260808 08:25:30.247138 3399 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [sys.catalog]: This master's current role is: LEADER
I20260808 08:25:30.247454 3401 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:25:30.249897 3401 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:25:30.250308 3401 catalog_manager.cc:1259] Loaded cluster ID: aafd38e6ed664f44b830a3f06aac46a6
I20260808 08:25:30.250353 3401 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:25:30.250898 3401 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:25:30.251277 3401 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971: Loaded TSK: 0
I20260808 08:25:30.251961 3401 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 08:25:30.277105 2168 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51868:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 08:25:30.280224 2168 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 08:25:30.283413 3397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:30.283160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46527 (local address 127.0.0.1:45598)
0808 08:25:30.283281 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:30.283342 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 08:25:30.283726 2494 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558 [term 3 FOLLOWER]: Refusing update from remote peer ad643f833e2a4adc89d9c6c143dbb971: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260808 08:25:30.284189 3397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:30.283967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46527 (local address 127.0.0.1:45604)
0808 08:25:30.284087 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:30.284137 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 08:25:30.284230 3399 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 42345 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260808 08:25:30.284436 2141 consensus_peers.cc:487] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 -> Peer 00ae12b3a32549bb9d8a3092fa6e0139 (127.22.235.254:46527): Couldn't send request to peer 00ae12b3a32549bb9d8a3092fa6e0139. Status: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46527: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 08:25:30.285780 3403 sys_catalog.cc:455] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad643f833e2a4adc89d9c6c143dbb971. Latest consensus state: current_term: 3 leader_uuid: "ad643f833e2a4adc89d9c6c143dbb971" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 42359 } attrs { promote: false } } peers { permanent_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 42345 } attrs { promote: false } } }
I20260808 08:25:30.285908 3403 sys_catalog.cc:458] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 08:25:30.286962 3399 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "ad643f833e2a4adc89d9c6c143dbb971" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 42359 } attrs { promote: false } } peers { permanent_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 42345 } attrs { promote: false } } }
I20260808 08:25:30.287081 3399 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971 [sys.catalog]: This master's current role is: LEADER
I20260808 08:25:30.287748 3403 sys_catalog.cc:455] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "ad643f833e2a4adc89d9c6c143dbb971" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "00ae12b3a32549bb9d8a3092fa6e0139" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46527 } } peers { permanent_uuid: "ad643f833e2a4adc89d9c6c143dbb971" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 42359 } attrs { promote: false } } peers { permanent_uuid: "b79f9c81ea0b4abeadd4ac1dbf54f558" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 42345 } attrs { promote: false } } }
I20260808 08:25:30.287928 3403 sys_catalog.cc:458] T 00000000000000000000000000000000 P b79f9c81ea0b4abeadd4ac1dbf54f558 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 08:25:30.297039 3413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:30.292647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54854 (local address 127.22.235.195:40217)
0808 08:25:30.293079 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:30.293084 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:30.293098 (+ 14us) server_negotiation.cc:407] Connection header received
0808 08:25:30.293147 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:30.293150 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:30.293202 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:30.293281 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:25:30.293801 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:30.294353 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:30.295704 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:30.296349 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:30.296462 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:30.296517 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:30.296802 (+ 285us) server_negotiation.cc:299] Negotiation successful
0808 08:25:30.296885 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":83,"threads_started":1}
I20260808 08:25:30.297041 3397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:30.292478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.195:40217 (local address 127.0.0.1:54854)
0808 08:25:30.292595 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:30.292609 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:30.292719 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:30.293573 (+ 854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:30.293576 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:30.293594 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:25:30.293697 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:30.293704 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:30.294461 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:30.294473 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:30.295598 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:30.295608 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:30.296457 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:30.296461 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:30.296669 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:30.296705 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 08:25:30.296778 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 08:25:30.296885 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":76}
I20260808 08:25:30.297043 3418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:30.293039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:39011 (local address 127.0.0.1:58530)
0808 08:25:30.293536 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:30.293549 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:30.293627 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:30.293898 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:30.293902 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:30.293922 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:25:30.294050 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:30.294057 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:30.294813 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:30.294816 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:30.295648 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:30.295658 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:30.296457 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:30.296461 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:30.296673 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:30.296704 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 08:25:30.296778 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 08:25:30.296885 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"mutex_wait_us":97,"thread_start_us":75,"threads_started":1}
I20260808 08:25:30.297514 3417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:30.293167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58530 (local address 127.22.235.193:39011)
0808 08:25:30.293424 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:30.293428 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:30.293614 (+ 186us) server_negotiation.cc:407] Connection header received
0808 08:25:30.293716 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:30.293719 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:30.293771 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:30.294139 (+ 368us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:25:30.294225 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:30.294708 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:30.295755 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:30.296349 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:30.296462 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:30.296517 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:30.297286 (+ 769us) server_negotiation.cc:299] Negotiation successful
0808 08:25:30.297375 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":89,"threads_started":1}
I20260808 08:25:30.297761 3121 tablet_service.cc:1467] Processing CreateTablet for tablet c7e8bc0b9c8c41dcb183ddb30e6fea8b (DEFAULT_TABLE table=test-table [id=1d4967766ce546ba99129907964f2633]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:25:30.298054 3121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7e8bc0b9c8c41dcb183ddb30e6fea8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:30.298040 2667 tablet_service.cc:1467] Processing CreateTablet for tablet c7e8bc0b9c8c41dcb183ddb30e6fea8b (DEFAULT_TABLE table=test-table [id=1d4967766ce546ba99129907964f2633]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:25:30.298318 2667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7e8bc0b9c8c41dcb183ddb30e6fea8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:30.300431 3416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:30.293302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.194:39211 (local address 127.0.0.1:40990)
0808 08:25:30.293883 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:30.293895 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:30.293963 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:30.294889 (+ 926us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:30.294892 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:30.294910 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:25:30.295049 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:30.295055 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:30.296907 (+ 1852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:30.296910 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:30.298243 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:30.298254 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:30.300037 (+ 1783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:30.300041 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:30.300138 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:30.300164 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 08:25:30.300234 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 08:25:30.300302 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"mutex_wait_us":26,"thread_start_us":84,"threads_started":1}
I20260808 08:25:30.300436 3421 tablet_bootstrap.cc:492] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 11de37660d0946beb33ae725e791d907: Bootstrap starting.
I20260808 08:25:30.300899 3420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:30.293636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40990 (local address 127.22.235.194:39211)
0808 08:25:30.294582 (+ 946us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:30.294587 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:30.294600 (+ 13us) server_negotiation.cc:407] Connection header received
0808 08:25:30.294649 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:30.294652 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:30.294697 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:30.294776 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:25:30.295178 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:30.296784 (+ 1606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:30.298455 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:30.299814 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:30.299926 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:30.299976 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:30.300674 (+ 698us) server_negotiation.cc:299] Negotiation successful
0808 08:25:30.300768 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":865,"thread_start_us":71,"threads_started":1}
I20260808 08:25:30.301415 3421 tablet_bootstrap.cc:654] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 11de37660d0946beb33ae725e791d907: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:30.301594 2901 tablet_service.cc:1467] Processing CreateTablet for tablet c7e8bc0b9c8c41dcb183ddb30e6fea8b (DEFAULT_TABLE table=test-table [id=1d4967766ce546ba99129907964f2633]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:25:30.301879 2901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7e8bc0b9c8c41dcb183ddb30e6fea8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:30.302130 3422 tablet_bootstrap.cc:492] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 27bb37319a6742ccb01e572b068b4bc9: Bootstrap starting.
I20260808 08:25:30.302680 3421 tablet_bootstrap.cc:492] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 11de37660d0946beb33ae725e791d907: No bootstrap required, opened a new log
I20260808 08:25:30.302775 3421 ts_tablet_manager.cc:1397] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 11de37660d0946beb33ae725e791d907: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 08:25:30.302995 3422 tablet_bootstrap.cc:654] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 27bb37319a6742ccb01e572b068b4bc9: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:30.303244 3421 raft_consensus.cc:348] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 11de37660d0946beb33ae725e791d907 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27bb37319a6742ccb01e572b068b4bc9" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 40217 } } peers { permanent_uuid: "11de37660d0946beb33ae725e791d907" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 39011 } } peers { permanent_uuid: "6b3a62b7e6a249d197410707a5bc428a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39211 } }
I20260808 08:25:30.303357 3421 raft_consensus.cc:374] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 11de37660d0946beb33ae725e791d907 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:30.303391 3421 raft_consensus.cc:729] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 11de37660d0946beb33ae725e791d907 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11de37660d0946beb33ae725e791d907, State: Initialized, Role: FOLLOWER
I20260808 08:25:30.303539 3421 consensus_queue.cc:260] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 11de37660d0946beb33ae725e791d907 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27bb37319a6742ccb01e572b068b4bc9" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 40217 } } peers { permanent_uuid: "11de37660d0946beb33ae725e791d907" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 39011 } } peers { permanent_uuid: "6b3a62b7e6a249d197410707a5bc428a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39211 } }
I20260808 08:25:30.303900 3421 ts_tablet_manager.cc:1428] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 11de37660d0946beb33ae725e791d907: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 08:25:30.304262 3422 tablet_bootstrap.cc:492] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 27bb37319a6742ccb01e572b068b4bc9: No bootstrap required, opened a new log
I20260808 08:25:30.304402 3422 ts_tablet_manager.cc:1397] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 27bb37319a6742ccb01e572b068b4bc9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 08:25:30.304795 3392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:30.304442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46527 (local address 127.22.235.193:47363)
0808 08:25:30.304677 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:30.304731 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151}
I20260808 08:25:30.304859 3422 raft_consensus.cc:348] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 27bb37319a6742ccb01e572b068b4bc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27bb37319a6742ccb01e572b068b4bc9" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 40217 } } peers { permanent_uuid: "11de37660d0946beb33ae725e791d907" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 39011 } } peers { permanent_uuid: "6b3a62b7e6a249d197410707a5bc428a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39211 } }
I20260808 08:25:30.304992 3422 raft_consensus.cc:374] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 27bb37319a6742ccb01e572b068b4bc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:30.305040 3422 raft_consensus.cc:729] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 27bb37319a6742ccb01e572b068b4bc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27bb37319a6742ccb01e572b068b4bc9, State: Initialized, Role: FOLLOWER
I20260808 08:25:30.305160 3422 consensus_queue.cc:260] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 27bb37319a6742ccb01e572b068b4bc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27bb37319a6742ccb01e572b068b4bc9" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 40217 } } peers { permanent_uuid: "11de37660d0946beb33ae725e791d907" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 39011 } } peers { permanent_uuid: "6b3a62b7e6a249d197410707a5bc428a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39211 } }
I20260808 08:25:30.306046 3391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:30.305818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46527 (local address 127.22.235.195:60089)
0808 08:25:30.305924 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:30.305974 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 08:25:30.306866 3430 tablet_bootstrap.cc:492] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 6b3a62b7e6a249d197410707a5bc428a: Bootstrap starting.
I20260808 08:25:30.307118 3422 ts_tablet_manager.cc:1428] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 27bb37319a6742ccb01e572b068b4bc9: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260808 08:25:30.307312 3245 heartbeater.cc:503] Master 127.22.235.253:42359 was elected leader, sending a full tablet report...
I20260808 08:25:30.307790 3430 tablet_bootstrap.cc:654] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 6b3a62b7e6a249d197410707a5bc428a: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:30.308212 3391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:30.307697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46527 (local address 127.22.235.195:38429)
0808 08:25:30.308099 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:30.308151 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342}
I20260808 08:25:30.308936 2793 heartbeater.cc:503] Master 127.22.235.253:42359 was elected leader, sending a full tablet report...
I20260808 08:25:30.309355 3392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:30.309134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46527 (local address 127.22.235.193:50695)
0808 08:25:30.309233 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:30.309294 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 08:25:30.310025 3430 tablet_bootstrap.cc:492] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 6b3a62b7e6a249d197410707a5bc428a: No bootstrap required, opened a new log
I20260808 08:25:30.310120 3430 ts_tablet_manager.cc:1397] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 6b3a62b7e6a249d197410707a5bc428a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 08:25:30.310454 3430 raft_consensus.cc:348] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 6b3a62b7e6a249d197410707a5bc428a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27bb37319a6742ccb01e572b068b4bc9" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 40217 } } peers { permanent_uuid: "11de37660d0946beb33ae725e791d907" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 39011 } } peers { permanent_uuid: "6b3a62b7e6a249d197410707a5bc428a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39211 } }
I20260808 08:25:30.310564 3430 raft_consensus.cc:374] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 6b3a62b7e6a249d197410707a5bc428a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:30.310585 3430 raft_consensus.cc:729] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 6b3a62b7e6a249d197410707a5bc428a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b3a62b7e6a249d197410707a5bc428a, State: Initialized, Role: FOLLOWER
I20260808 08:25:30.310675 3430 consensus_queue.cc:260] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 6b3a62b7e6a249d197410707a5bc428a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27bb37319a6742ccb01e572b068b4bc9" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 40217 } } peers { permanent_uuid: "11de37660d0946beb33ae725e791d907" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 39011 } } peers { permanent_uuid: "6b3a62b7e6a249d197410707a5bc428a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39211 } }
I20260808 08:25:30.310935 3430 ts_tablet_manager.cc:1428] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 6b3a62b7e6a249d197410707a5bc428a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 08:25:30.311498 3394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:30.311257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46527 (local address 127.22.235.194:43763)
0808 08:25:30.311367 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:30.311421 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 08:25:30.312364 3009 heartbeater.cc:503] Master 127.22.235.253:42359 was elected leader, sending a full tablet report...
I20260808 08:25:30.312696 3394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:30.312394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46527 (local address 127.22.235.194:51107)
0808 08:25:30.312559 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:30.312606 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 08:25:30.449703 3431 raft_consensus.cc:482] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 27bb37319a6742ccb01e572b068b4bc9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 08:25:30.449860 3431 raft_consensus.cc:504] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 27bb37319a6742ccb01e572b068b4bc9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27bb37319a6742ccb01e572b068b4bc9" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 40217 } } peers { permanent_uuid: "11de37660d0946beb33ae725e791d907" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 39011 } } peers { permanent_uuid: "6b3a62b7e6a249d197410707a5bc428a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39211 } }
I20260808 08:25:30.450981 3431 leader_election.cc:288] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 27bb37319a6742ccb01e572b068b4bc9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 11de37660d0946beb33ae725e791d907 (127.22.235.193:39011), 6b3a62b7e6a249d197410707a5bc428a (127.22.235.194:39211)
I20260808 08:25:30.454677 3391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:30.451242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:39011 (local address 127.22.235.195:45943)
0808 08:25:30.451389 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:30.451404 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:30.451566 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:30.451879 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:30.451882 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:30.451903 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:25:30.452016 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:30.452021 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:30.452740 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:30.452744 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:30.453551 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:30.453560 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:30.454240 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:30.454243 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:30.454345 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:30.454392 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 08:25:30.454462 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 08:25:30.454536 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 08:25:30.454685 3417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:30.451364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:45943 (local address 127.22.235.193:39011)
0808 08:25:30.451500 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:30.451505 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:30.451524 (+ 19us) server_negotiation.cc:407] Connection header received
0808 08:25:30.451701 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:30.451705 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:30.451767 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:30.451857 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:25:30.452114 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:30.452637 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:30.453671 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:30.454133 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:30.454239 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:30.454294 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:30.454480 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 08:25:30.454565 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 08:25:30.454939 3437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:30.451518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.194:39211 (local address 127.22.235.195:51883)
0808 08:25:30.451867 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:30.451878 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:30.451977 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:30.452192 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:30.452194 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:30.452209 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:25:30.452339 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:30.452345 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:30.453101 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:30.453105 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:30.453845 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:30.453854 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:30.454517 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:30.454521 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:30.454620 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:30.454637 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 08:25:30.454726 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 08:25:30.454793 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":131,"threads_started":1}
I20260808 08:25:30.454934 3420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:30.451667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:51883 (local address 127.22.235.194:39211)
0808 08:25:30.451806 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:30.451812 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:30.451950 (+ 138us) server_negotiation.cc:407] Connection header received
0808 08:25:30.452065 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:30.452068 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:30.452112 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:30.452173 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:25:30.452446 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:30.452994 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:30.453971 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:30.454378 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:30.454487 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:30.454538 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:30.454746 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 08:25:30.454834 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 08:25:30.455368 2709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7e8bc0b9c8c41dcb183ddb30e6fea8b" candidate_uuid: "27bb37319a6742ccb01e572b068b4bc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11de37660d0946beb33ae725e791d907" is_pre_election: true
I20260808 08:25:30.455539 2709 raft_consensus.cc:2454] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 11de37660d0946beb33ae725e791d907 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27bb37319a6742ccb01e572b068b4bc9 in term 0.
I20260808 08:25:30.455677 2942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7e8bc0b9c8c41dcb183ddb30e6fea8b" candidate_uuid: "27bb37319a6742ccb01e572b068b4bc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b3a62b7e6a249d197410707a5bc428a" is_pre_election: true
I20260808 08:25:30.455917 2942 raft_consensus.cc:2454] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 6b3a62b7e6a249d197410707a5bc428a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27bb37319a6742ccb01e572b068b4bc9 in term 0.
I20260808 08:25:30.455917 3045 leader_election.cc:302] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 27bb37319a6742ccb01e572b068b4bc9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11de37660d0946beb33ae725e791d907, 27bb37319a6742ccb01e572b068b4bc9; no voters:
I20260808 08:25:30.456203 3431 raft_consensus.cc:2781] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 27bb37319a6742ccb01e572b068b4bc9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 08:25:30.456262 3431 raft_consensus.cc:482] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 27bb37319a6742ccb01e572b068b4bc9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 08:25:30.456312 3431 raft_consensus.cc:3037] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 27bb37319a6742ccb01e572b068b4bc9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:30.457351 3431 raft_consensus.cc:504] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 27bb37319a6742ccb01e572b068b4bc9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27bb37319a6742ccb01e572b068b4bc9" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 40217 } } peers { permanent_uuid: "11de37660d0946beb33ae725e791d907" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 39011 } } peers { permanent_uuid: "6b3a62b7e6a249d197410707a5bc428a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39211 } }
I20260808 08:25:30.457697 3431 leader_election.cc:288] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 27bb37319a6742ccb01e572b068b4bc9 [CANDIDATE]: Term 1 election: Requested vote from peers 11de37660d0946beb33ae725e791d907 (127.22.235.193:39011), 6b3a62b7e6a249d197410707a5bc428a (127.22.235.194:39211)
I20260808 08:25:30.458050 2709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7e8bc0b9c8c41dcb183ddb30e6fea8b" candidate_uuid: "27bb37319a6742ccb01e572b068b4bc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11de37660d0946beb33ae725e791d907"
I20260808 08:25:30.458102 2942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7e8bc0b9c8c41dcb183ddb30e6fea8b" candidate_uuid: "27bb37319a6742ccb01e572b068b4bc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b3a62b7e6a249d197410707a5bc428a"
I20260808 08:25:30.458173 2709 raft_consensus.cc:3037] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 11de37660d0946beb33ae725e791d907 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:30.458199 2942 raft_consensus.cc:3037] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 6b3a62b7e6a249d197410707a5bc428a [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:30.459281 2709 raft_consensus.cc:2454] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 11de37660d0946beb33ae725e791d907 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27bb37319a6742ccb01e572b068b4bc9 in term 1.
I20260808 08:25:30.459281 2942 raft_consensus.cc:2454] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 6b3a62b7e6a249d197410707a5bc428a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27bb37319a6742ccb01e572b068b4bc9 in term 1.
I20260808 08:25:30.459549 3045 leader_election.cc:302] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 27bb37319a6742ccb01e572b068b4bc9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27bb37319a6742ccb01e572b068b4bc9, 6b3a62b7e6a249d197410707a5bc428a; no voters:
I20260808 08:25:30.459705 3431 raft_consensus.cc:2781] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 27bb37319a6742ccb01e572b068b4bc9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:25:30.459929 3431 raft_consensus.cc:686] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 27bb37319a6742ccb01e572b068b4bc9 [term 1 LEADER]: Becoming Leader. State: Replica: 27bb37319a6742ccb01e572b068b4bc9, State: Running, Role: LEADER
I20260808 08:25:30.460381 3431 consensus_queue.cc:237] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 27bb37319a6742ccb01e572b068b4bc9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27bb37319a6742ccb01e572b068b4bc9" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 40217 } } peers { permanent_uuid: "11de37660d0946beb33ae725e791d907" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 39011 } } peers { permanent_uuid: "6b3a62b7e6a249d197410707a5bc428a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39211 } }
I20260808 08:25:30.460654 3437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:30.460393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46527 (local address 127.22.235.195:60237)
0808 08:25:30.460505 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:30.460558 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 08:25:30.462819 2168 catalog_manager.cc:5495] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 27bb37319a6742ccb01e572b068b4bc9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 27bb37319a6742ccb01e572b068b4bc9 (127.22.235.195). New cstate: current_term: 1 leader_uuid: "27bb37319a6742ccb01e572b068b4bc9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27bb37319a6742ccb01e572b068b4bc9" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 40217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11de37660d0946beb33ae725e791d907" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 39011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b3a62b7e6a249d197410707a5bc428a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39211 } health_report { overall_health: UNKNOWN } } }
W20260808 08:25:30.466974 3247 tablet.cc:2369] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 27bb37319a6742ccb01e572b068b4bc9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 08:25:30.483572 3010 tablet.cc:2369] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 6b3a62b7e6a249d197410707a5bc428a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 08:25:30.484608 3413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:30.480843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54866 (local address 127.22.235.195:40217)
0808 08:25:30.480940 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:30.480944 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:30.481068 (+ 124us) server_negotiation.cc:407] Connection header received
0808 08:25:30.481236 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:30.481240 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:30.481295 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:30.481382 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:25:30.481626 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:30.482193 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:30.483040 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:30.483256 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:30.483340 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:30.483664 (+ 324us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:25:30.484211 (+ 547us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 08:25:30.484322 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:30.484403 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 08:25:30.484482 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 08:25:30.487504 2709 raft_consensus.cc:1261] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 11de37660d0946beb33ae725e791d907 [term 1 FOLLOWER]: Refusing update from remote peer 27bb37319a6742ccb01e572b068b4bc9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:25:30.487773 2942 raft_consensus.cc:1261] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 6b3a62b7e6a249d197410707a5bc428a [term 1 FOLLOWER]: Refusing update from remote peer 27bb37319a6742ccb01e572b068b4bc9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:25:30.488046 3431 consensus_queue.cc:1035] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 27bb37319a6742ccb01e572b068b4bc9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11de37660d0946beb33ae725e791d907" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 39011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:25:30.488344 3440 consensus_queue.cc:1035] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 27bb37319a6742ccb01e572b068b4bc9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b3a62b7e6a249d197410707a5bc428a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:25:30.490856 3394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:30.490600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46527 (local address 127.22.235.194:44175)
0808 08:25:30.490723 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:30.490788 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 08:25:30.491328 3392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:30.490274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46527 (local address 127.22.235.193:40521)
0808 08:25:30.491201 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:30.491256 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":856}
I20260808 08:25:30.492395 3447 mvcc.cc:205] Tried to move back new op lower bound from 7316183164871573504 to 7316183164768669696. Current Snapshot: MvccSnapshot[applied={T|T < 7316183164871573504}]
I20260808 08:25:30.492791 3394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:30.492501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46527 (local address 127.22.235.194:35273)
0808 08:25:30.492675 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:30.492729 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260808 08:25:30.493448 3437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:30.493229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46527 (local address 127.22.235.195:36303)
0808 08:25:30.493331 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:30.493382 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 08:25:30.495484 3437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:30.495257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46527 (local address 127.22.235.195:59951)
0808 08:25:30.495367 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:30.495420 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 08:25:30.501178 3392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:30.500902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46527 (local address 127.22.235.193:38899)
0808 08:25:30.501050 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:30.501107 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260808 08:25:30.501868 2168 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51868:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:25:30.502020 2168 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51868:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:25:30.505837 2168 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ad643f833e2a4adc89d9c6c143dbb971: Sending DeleteTablet for 3 replicas of tablet c7e8bc0b9c8c41dcb183ddb30e6fea8b
I20260808 08:25:30.506441 3121 tablet_service.cc:1514] Processing DeleteTablet for tablet c7e8bc0b9c8c41dcb183ddb30e6fea8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:25:30 UTC) from {username='oryx', principal='oryx/127.22.235.253@KRBTEST.COM'} at 127.0.0.1:54854
I20260808 08:25:30.506861 2667 tablet_service.cc:1514] Processing DeleteTablet for tablet c7e8bc0b9c8c41dcb183ddb30e6fea8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:25:30 UTC) from {username='oryx', principal='oryx/127.22.235.253@KRBTEST.COM'} at 127.0.0.1:58530
I20260808 08:25:30.506870 2901 tablet_service.cc:1514] Processing DeleteTablet for tablet c7e8bc0b9c8c41dcb183ddb30e6fea8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:25:30 UTC) from {username='oryx', principal='oryx/127.22.235.253@KRBTEST.COM'} at 127.0.0.1:40990
I20260808 08:25:30.507315 3461 tablet_replica.cc:330] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 11de37660d0946beb33ae725e791d907: stopping tablet replica
I20260808 08:25:30.507424 3460 tablet_replica.cc:330] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 27bb37319a6742ccb01e572b068b4bc9: stopping tablet replica
I20260808 08:25:30.507479 3461 raft_consensus.cc:2229] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 11de37660d0946beb33ae725e791d907 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 08:25:30.507567 3460 raft_consensus.cc:2229] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 27bb37319a6742ccb01e572b068b4bc9 [term 1 LEADER]: Raft consensus shutting down.
I20260808 08:25:30.507696 3461 raft_consensus.cc:2258] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 11de37660d0946beb33ae725e791d907 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:25:30.507720 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 2411
I20260808 08:25:30.507864 3460 raft_consensus.cc:2258] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 27bb37319a6742ccb01e572b068b4bc9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:25:30.508488 3465 tablet_replica.cc:330] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 6b3a62b7e6a249d197410707a5bc428a: stopping tablet replica
I20260808 08:25:30.508602 3465 raft_consensus.cc:2229] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 6b3a62b7e6a249d197410707a5bc428a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 08:25:30.508747 3465 raft_consensus.cc:2258] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 6b3a62b7e6a249d197410707a5bc428a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:25:30.511608 3465 ts_tablet_manager.cc:1905] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 6b3a62b7e6a249d197410707a5bc428a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:25:30.514540 3465 ts_tablet_manager.cc:1918] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 6b3a62b7e6a249d197410707a5bc428a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 08:25:30.516368 3465 log.cc:1196] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 6b3a62b7e6a249d197410707a5bc428a: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-1/wal/wals/c7e8bc0b9c8c41dcb183ddb30e6fea8b
I20260808 08:25:30.516728 3465 ts_tablet_manager.cc:1939] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 6b3a62b7e6a249d197410707a5bc428a: Deleting consensus metadata
I20260808 08:25:30.517030 3460 ts_tablet_manager.cc:1905] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 27bb37319a6742ccb01e572b068b4bc9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:25:30.519109 3460 ts_tablet_manager.cc:1918] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 27bb37319a6742ccb01e572b068b4bc9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 08:25:30.519179 3460 log.cc:1196] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 27bb37319a6742ccb01e572b068b4bc9: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786177493204855-23471-0/minicluster-data/ts-2/wal/wals/c7e8bc0b9c8c41dcb183ddb30e6fea8b
I20260808 08:25:30.519397 3460 ts_tablet_manager.cc:1939] T c7e8bc0b9c8c41dcb183ddb30e6fea8b P 27bb37319a6742ccb01e572b068b4bc9: Deleting consensus metadata
I20260808 08:25:30.520176 2146 catalog_manager.cc:4841] TS 27bb37319a6742ccb01e572b068b4bc9 (127.22.235.195:40217): tablet c7e8bc0b9c8c41dcb183ddb30e6fea8b (table test-table [id=1d4967766ce546ba99129907964f2633]) successfully deleted
I20260808 08:25:30.520831 2141 catalog_manager.cc:4841] TS 6b3a62b7e6a249d197410707a5bc428a (127.22.235.194:39211): tablet c7e8bc0b9c8c41dcb183ddb30e6fea8b (table test-table [id=1d4967766ce546ba99129907964f2633]) successfully deleted
I20260808 08:25:30.524897 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 2806
I20260808 08:25:30.525133 3416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:30.524856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:39011 (local address 127.0.0.1:58538)
0808 08:25:30.524994 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:30.525047 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.193:39011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260808 08:25:30.532903 2141 catalog_manager.cc:4584] TS 11de37660d0946beb33ae725e791d907 (127.22.235.193:39011): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c7e8bc0b9c8c41dcb183ddb30e6fea8b: Network error: Client connection negotiation failed: client connection to 127.22.235.193:39011: connect: Connection refused (error 111)
I20260808 08:25:30.542774 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 3016
I20260808 08:25:30.560642 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 1925
I20260808 08:25:30.572722 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 2122
2026-08-08T08:25:30Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7328 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 08:25:30 dist-test-slave-f7th krb5kdc[3469](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 08:25:30 dist-test-slave-f7th krb5kdc[3469](info): set up 2 sockets
Aug 08 08:25:30 dist-test-slave-f7th krb5kdc[3469](info): commencing operation
krb5kdc: starting...
W20260808 08:25:31.738850 23471 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Aug 08 08:25:31 dist-test-slave-f7th krb5kdc[3469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177531, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T08:25:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:25:31Z Disabled control of system clock
WARNING: no policy specified for oryx/127.22.235.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.235.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.235.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.254.keytab.
Entry for principal oryx/127.22.235.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.254.keytab.
I20260808 08:25:31.855196 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:36595
--webserver_interface=127.22.235.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.254.keytab
--principal=oryx/127.22.235.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:38637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:36595
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:25:31.966157 3492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:31.966400 3492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:31.966439 3492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:31.970289 3492 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:25:31.970335 3492 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 08:25:31.970350 3492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:31.970366 3492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:31.970381 3492 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:25:31.970400 3492 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:25:31.975415 3492 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:38637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:36595
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.254.keytab
--principal=oryx/127.22.235.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:36595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.235.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:31.976325 3492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:31.977267 3492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 08:25:31.985211 3492 server_base.cc:1034] running on GCE node
W20260808 08:25:31.985177 3501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:31.985167 3500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:31.985177 3504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:31.985954 3492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:31.987036 3492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:31.988193 3492 hybrid_clock.cc:648] HybridClock initialized: now 1786177531988178 us; error 29 us; skew 500 ppm
Aug 08 08:25:31 dist-test-slave-f7th krb5kdc[3469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177531, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.235.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:31.990854 3492 init.cc:377] Logged in from keytab as oryx/127.22.235.254@KRBTEST.COM (short username oryx)
I20260808 08:25:31.991719 3492 webserver.cc:460] Webserver started at http://127.22.235.254:44735/ using document root <none> and password file <none>
I20260808 08:25:31.992238 3492 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:31.992303 3492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:31.992488 3492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:31.994462 3492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "a626e0cf38834d9992e58716fe7ae506"
format_stamp: "Formatted at 2026-08-08 08:25:31 on dist-test-slave-f7th"
server_key: "45e4a23dcc027b2cc3f816062808cd4d"
server_key_iv: "b8be4e4e52438240b6e9497386ef293d"
server_key_version: "encryptionkey@0"
I20260808 08:25:31.994990 3492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "a626e0cf38834d9992e58716fe7ae506"
format_stamp: "Formatted at 2026-08-08 08:25:31 on dist-test-slave-f7th"
server_key: "45e4a23dcc027b2cc3f816062808cd4d"
server_key_iv: "b8be4e4e52438240b6e9497386ef293d"
server_key_version: "encryptionkey@0"
I20260808 08:25:31.998456 3492 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 08:25:32.000793 3519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:32.001979 3492 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:25:32.002094 3492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "a626e0cf38834d9992e58716fe7ae506"
format_stamp: "Formatted at 2026-08-08 08:25:31 on dist-test-slave-f7th"
server_key: "45e4a23dcc027b2cc3f816062808cd4d"
server_key_iv: "b8be4e4e52438240b6e9497386ef293d"
server_key_version: "encryptionkey@0"
I20260808 08:25:32.002182 3492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:32.023213 3492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:32.025943 3492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:32.026121 3492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:32.033799 3492 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.254:36595
I20260808 08:25:32.033838 3622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.254:36595 every 8 connection(s)
I20260808 08:25:32.034698 3492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:25:32.037433 3624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:32.040911 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 3492
I20260808 08:25:32.041033 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:25:32.041292 23471 external_mini_cluster.cc:1428] Setting key 6fce8817e6285106e9d23c2c0222e767
I20260808 08:25:32.043201 3624 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a626e0cf38834d9992e58716fe7ae506: Bootstrap starting.
I20260808 08:25:32.045665 3624 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a626e0cf38834d9992e58716fe7ae506: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:32.046370 3624 log.cc:824] T 00000000000000000000000000000000 P a626e0cf38834d9992e58716fe7ae506: Log is configured to *not* fsync() on all Append() calls
I20260808 08:25:32.048483 3624 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a626e0cf38834d9992e58716fe7ae506: No bootstrap required, opened a new log
Aug 08 08:25:32 dist-test-slave-f7th krb5kdc[3469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177531, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM
I20260808 08:25:32.051914 3624 raft_consensus.cc:348] T 00000000000000000000000000000000 P a626e0cf38834d9992e58716fe7ae506 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a626e0cf38834d9992e58716fe7ae506" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 36595 } }
I20260808 08:25:32.052120 3624 raft_consensus.cc:374] T 00000000000000000000000000000000 P a626e0cf38834d9992e58716fe7ae506 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:32.052156 3624 raft_consensus.cc:729] T 00000000000000000000000000000000 P a626e0cf38834d9992e58716fe7ae506 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a626e0cf38834d9992e58716fe7ae506, State: Initialized, Role: FOLLOWER
I20260808 08:25:32.052598 3624 consensus_queue.cc:260] T 00000000000000000000000000000000 P a626e0cf38834d9992e58716fe7ae506 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a626e0cf38834d9992e58716fe7ae506" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 36595 } }
I20260808 08:25:32.052753 3624 raft_consensus.cc:388] T 00000000000000000000000000000000 P a626e0cf38834d9992e58716fe7ae506 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:25:32.052816 3624 raft_consensus.cc:482] T 00000000000000000000000000000000 P a626e0cf38834d9992e58716fe7ae506 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:25:32.052904 3624 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a626e0cf38834d9992e58716fe7ae506 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:32.053723 3624 raft_consensus.cc:504] T 00000000000000000000000000000000 P a626e0cf38834d9992e58716fe7ae506 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a626e0cf38834d9992e58716fe7ae506" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 36595 } }
I20260808 08:25:32.054082 3624 leader_election.cc:302] T 00000000000000000000000000000000 P a626e0cf38834d9992e58716fe7ae506 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a626e0cf38834d9992e58716fe7ae506; no voters:
I20260808 08:25:32.054387 3624 leader_election.cc:288] T 00000000000000000000000000000000 P a626e0cf38834d9992e58716fe7ae506 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:25:32.054466 3632 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a626e0cf38834d9992e58716fe7ae506 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:25:32.054675 3632 raft_consensus.cc:686] T 00000000000000000000000000000000 P a626e0cf38834d9992e58716fe7ae506 [term 1 LEADER]: Becoming Leader. State: Replica: a626e0cf38834d9992e58716fe7ae506, State: Running, Role: LEADER
I20260808 08:25:32.055030 3632 consensus_queue.cc:237] T 00000000000000000000000000000000 P a626e0cf38834d9992e58716fe7ae506 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a626e0cf38834d9992e58716fe7ae506" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 36595 } }
I20260808 08:25:32.055194 3627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:32.042634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53838 (local address 127.22.235.254:36595)
0808 08:25:32.043030 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:32.043045 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:32.043082 (+ 37us) server_negotiation.cc:407] Connection header received
0808 08:25:32.043700 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:32.043718 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:32.044061 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:32.044328 (+ 267us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:32.045208 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.046246 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.047234 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.047698 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.047818 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:32.050538 (+ 2720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:32.050543 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:32.050556 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:32.050583 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:32.052107 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:32.052514 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:32.052519 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:32.052524 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:32.052679 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:32.052993 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:32.052997 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:32.052998 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:32.053367 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:32.053554 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:32.053843 (+ 289us) server_negotiation.cc:299] Negotiation successful
0808 08:25:32.054121 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":96,"threads_started":1}
I20260808 08:25:32.055668 3624 sys_catalog.cc:564] T 00000000000000000000000000000000 P a626e0cf38834d9992e58716fe7ae506 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:25:32.057255 3634 sys_catalog.cc:455] T 00000000000000000000000000000000 P a626e0cf38834d9992e58716fe7ae506 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a626e0cf38834d9992e58716fe7ae506" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a626e0cf38834d9992e58716fe7ae506" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 36595 } } }
I20260808 08:25:32.057377 3634 sys_catalog.cc:458] T 00000000000000000000000000000000 P a626e0cf38834d9992e58716fe7ae506 [sys.catalog]: This master's current role is: LEADER
I20260808 08:25:32.057587 3643 sys_catalog.cc:455] T 00000000000000000000000000000000 P a626e0cf38834d9992e58716fe7ae506 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a626e0cf38834d9992e58716fe7ae506. Latest consensus state: current_term: 1 leader_uuid: "a626e0cf38834d9992e58716fe7ae506" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a626e0cf38834d9992e58716fe7ae506" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 36595 } } }
I20260808 08:25:32.057689 3643 sys_catalog.cc:458] T 00000000000000000000000000000000 P a626e0cf38834d9992e58716fe7ae506 [sys.catalog]: This master's current role is: LEADER
W20260808 08:25:32.060251 3661 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a626e0cf38834d9992e58716fe7ae506: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 08:25:32.060314 3661 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 08:25:32.060601 3660 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:25:32.061269 3660 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:25:32.067062 3660 catalog_manager.cc:1347] Generated new cluster ID: d90b5c5adeba43369d4ca203945c61ff
I20260808 08:25:32.067129 3660 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:25:32.078344 3660 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:25:32.079245 3660 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:25:32.087815 3660 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a626e0cf38834d9992e58716fe7ae506: Generated new TSK 0
I20260808 08:25:32.088563 3660 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.22.235.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.235.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.235.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.193.keytab.
Entry for principal oryx/127.22.235.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.193.keytab.
I20260808 08:25:32.121248 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.193:0
--local_ip_for_outbound_sockets=127.22.235.193
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:36595
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.193.keytab
--principal=oryx/127.22.235.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:38637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:25:32.231866 3673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:32.232117 3673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:32.232161 3673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:32.236053 3673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:32.236112 3673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:32.236151 3673 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 08:25:32.236189 3673 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.193
I20260808 08:25:32.241212 3673 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:38637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.193.keytab
--principal=oryx/127.22.235.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.235.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.235.254:36595
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3673
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.193
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:32.242141 3673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:32.243163 3673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:32.252409 3683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:32.252413 3681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:32.252441 3686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:32.252760 3673 server_base.cc:1034] running on GCE node
I20260808 08:25:32.253183 3673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:32.253839 3673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:32.255046 3673 hybrid_clock.cc:648] HybridClock initialized: now 1786177532255022 us; error 45 us; skew 500 ppm
Aug 08 08:25:32 dist-test-slave-f7th krb5kdc[3469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177532, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.235.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:32.257997 3673 init.cc:377] Logged in from keytab as oryx/127.22.235.193@KRBTEST.COM (short username oryx)
I20260808 08:25:32.258857 3673 webserver.cc:460] Webserver started at http://127.22.235.193:41927/ using document root <none> and password file <none>
I20260808 08:25:32.259404 3673 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:32.259472 3673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:32.259667 3673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:32.261710 3673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-0/data/instance:
uuid: "bc0b21918b524c05aeada9beda0f7b75"
format_stamp: "Formatted at 2026-08-08 08:25:32 on dist-test-slave-f7th"
server_key: "0395ff1a7a5bb1fdc6f22595a2de5919"
server_key_iv: "84aaaeb6f7d3701364ba01a58a90dcae"
server_key_version: "encryptionkey@0"
I20260808 08:25:32.262252 3673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance:
uuid: "bc0b21918b524c05aeada9beda0f7b75"
format_stamp: "Formatted at 2026-08-08 08:25:32 on dist-test-slave-f7th"
server_key: "0395ff1a7a5bb1fdc6f22595a2de5919"
server_key_iv: "84aaaeb6f7d3701364ba01a58a90dcae"
server_key_version: "encryptionkey@0"
I20260808 08:25:32.265865 3673 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 08:25:32.268201 3698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:32.269397 3673 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:25:32.269505 3673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-0/wal
uuid: "bc0b21918b524c05aeada9beda0f7b75"
format_stamp: "Formatted at 2026-08-08 08:25:32 on dist-test-slave-f7th"
server_key: "0395ff1a7a5bb1fdc6f22595a2de5919"
server_key_iv: "84aaaeb6f7d3701364ba01a58a90dcae"
server_key_version: "encryptionkey@0"
I20260808 08:25:32.269595 3673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:32.282825 3673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:32.285998 3673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:32.286180 3673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:32.287644 3673 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:25:32.287694 3673 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:32.287748 3673 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:25:32.287779 3673 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 08:25:32 dist-test-slave-f7th krb5kdc[3469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177532, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.235.193@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM
I20260808 08:25:32.298202 3673 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.193:46367
I20260808 08:25:32.298318 3908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.193:46367 every 8 connection(s)
I20260808 08:25:32.299283 3673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
I20260808 08:25:32.301373 3627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:32.288371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:36863 (local address 127.22.235.254:36595)
0808 08:25:32.288480 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:32.288483 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:32.289530 (+ 1047us) server_negotiation.cc:407] Connection header received
0808 08:25:32.290454 (+ 924us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:32.290458 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:32.290515 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:32.290590 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:32.292021 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.292880 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.294519 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.294881 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.294998 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:32.297779 (+ 2781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:32.297784 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:32.297786 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:32.297817 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:32.299403 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:32.299939 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:32.299943 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:32.299945 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:32.300046 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:32.300354 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:32.300357 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:32.300359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:32.300536 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:32.300641 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:32.301116 (+ 475us) server_negotiation.cc:299] Negotiation successful
0808 08:25:32.301225 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 08:25:32.302045 3721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:32.288925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:36595 (local address 127.22.235.193:36863)
0808 08:25:32.289390 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:32.289429 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:32.290245 (+ 816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:32.290776 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:32.290785 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:32.291205 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:32.291841 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:32.291861 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.293021 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.293025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:32.294373 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:32.294385 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.295040 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.295045 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:32.295242 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:32.295808 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:32.295824 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:32.297610 (+ 1786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:32.299511 (+ 1901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:32.299516 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:32.299527 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:32.299787 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:32.300133 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:32.300136 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:32.300138 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:32.300267 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:32.300650 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:32.300656 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:32.300875 (+ 219us) client_negotiation.cc:769] Sending connection context
0808 08:25:32.301037 (+ 162us) client_negotiation.cc:240] Negotiation successful
0808 08:25:32.301274 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":130,"threads_started":1}
I20260808 08:25:32.303256 3909 heartbeater.cc:347] Connected to a master server at 127.22.235.254:36595
I20260808 08:25:32.303393 3909 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:32.303908 3909 heartbeater.cc:511] Master 127.22.235.254:36595 requested a full tablet report, sending...
I20260808 08:25:32.305554 3545 ts_manager.cc:194] Registered new tserver with Master: bc0b21918b524c05aeada9beda0f7b75 (127.22.235.193:46367)
I20260808 08:25:32.306898 3545 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.235.193@KRBTEST.COM'} at 127.22.235.193:36863
I20260808 08:25:32.307173 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 3673
I20260808 08:25:32.307302 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance
I20260808 08:25:32.307638 23471 external_mini_cluster.cc:1428] Setting key 29bfd53050719bd7ecd80fbf88f47333
I20260808 08:25:32.312039 3627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:32.305460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:35505 (local address 127.22.235.254:36595)
0808 08:25:32.305585 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:32.305589 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:32.305762 (+ 173us) server_negotiation.cc:407] Connection header received
0808 08:25:32.305878 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:32.305881 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:32.305924 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:32.306012 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:32.306429 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.306918 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.307955 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.308230 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.308323 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:32.309460 (+ 1137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:32.309463 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:32.309465 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:32.309490 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:32.310596 (+ 1106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:32.310984 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:32.310988 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:32.310989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:32.311064 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:32.311306 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:32.311309 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:32.311311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:32.311470 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:32.311552 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:32.311794 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 08:25:32.311891 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 08:25:32.312045 3916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:32.305398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:36595 (local address 127.22.235.193:35505)
0808 08:25:32.305683 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:32.305697 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:32.305791 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:32.306037 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:32.306040 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:32.306230 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:32.306336 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:32.306343 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.307084 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.307087 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:32.307823 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:32.307834 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.308421 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.308423 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:32.308505 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:32.308925 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:32.308939 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:32.309319 (+ 380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:32.310697 (+ 1378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:32.310700 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:32.310702 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:32.310891 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:32.311147 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:32.311150 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:32.311152 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:32.311229 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:32.311558 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:32.311562 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:32.311646 (+ 84us) client_negotiation.cc:769] Sending connection context
0808 08:25:32.311730 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 08:25:32.311856 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":1,"thread_start_us":120,"threads_started":1}
WARNING: no policy specified for oryx/127.22.235.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.235.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.235.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.194.keytab.
Entry for principal oryx/127.22.235.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.194.keytab.
I20260808 08:25:32.333067 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.194:0
--local_ip_for_outbound_sockets=127.22.235.194
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:36595
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.194.keytab
--principal=oryx/127.22.235.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:38637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:25:32.443967 3919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:32.444221 3919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:32.444267 3919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:32.448138 3919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:32.448190 3919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:32.448233 3919 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 08:25:32.448284 3919 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.194
I20260808 08:25:32.453274 3919 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:38637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.194.keytab
--principal=oryx/127.22.235.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.235.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.235.254:36595
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.194
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:32.454223 3919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:32.455216 3919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:32.464496 3933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:32.464550 3931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:32.464507 3930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:32.464735 3919 server_base.cc:1034] running on GCE node
I20260808 08:25:32.465193 3919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:32.465787 3919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:32.466939 3919 hybrid_clock.cc:648] HybridClock initialized: now 1786177532466919 us; error 31 us; skew 500 ppm
Aug 08 08:25:32 dist-test-slave-f7th krb5kdc[3469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177532, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.235.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:32.469591 3919 init.cc:377] Logged in from keytab as oryx/127.22.235.194@KRBTEST.COM (short username oryx)
I20260808 08:25:32.470397 3919 webserver.cc:460] Webserver started at http://127.22.235.194:38261/ using document root <none> and password file <none>
I20260808 08:25:32.470911 3919 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:32.470973 3919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:32.471139 3919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:32.473125 3919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-1/data/instance:
uuid: "417a70ae534e48e580b4dad4ae39dde6"
format_stamp: "Formatted at 2026-08-08 08:25:32 on dist-test-slave-f7th"
server_key: "fbb963bfd3bf381e4c4f14b7d8494012"
server_key_iv: "8511fc1f50ecda4563caa9663ab78a0f"
server_key_version: "encryptionkey@0"
I20260808 08:25:32.473685 3919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance:
uuid: "417a70ae534e48e580b4dad4ae39dde6"
format_stamp: "Formatted at 2026-08-08 08:25:32 on dist-test-slave-f7th"
server_key: "fbb963bfd3bf381e4c4f14b7d8494012"
server_key_iv: "8511fc1f50ecda4563caa9663ab78a0f"
server_key_version: "encryptionkey@0"
I20260808 08:25:32.477191 3919 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 08:25:32.479575 3943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:32.480779 3919 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:25:32.480886 3919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-1/wal
uuid: "417a70ae534e48e580b4dad4ae39dde6"
format_stamp: "Formatted at 2026-08-08 08:25:32 on dist-test-slave-f7th"
server_key: "fbb963bfd3bf381e4c4f14b7d8494012"
server_key_iv: "8511fc1f50ecda4563caa9663ab78a0f"
server_key_version: "encryptionkey@0"
I20260808 08:25:32.480991 3919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:32.516019 3919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:32.519234 3919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:32.519419 3919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:32.520979 3919 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:25:32.521034 3919 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:32.521088 3919 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:25:32.521117 3919 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 08:25:32 dist-test-slave-f7th krb5kdc[3469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177532, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.235.194@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM
I20260808 08:25:32.531335 3919 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.194:34595
I20260808 08:25:32.531482 4150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.194:34595 every 8 connection(s)
I20260808 08:25:32.532328 3919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
I20260808 08:25:32.533921 3627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:32.521648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:42463 (local address 127.22.235.254:36595)
0808 08:25:32.521795 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:32.521800 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:32.522681 (+ 881us) server_negotiation.cc:407] Connection header received
0808 08:25:32.523719 (+ 1038us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:32.523724 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:32.523783 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:32.523841 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:32.525374 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.526004 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.527108 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.527436 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.527536 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:32.530402 (+ 2866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:32.530405 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:32.530407 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:32.530432 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:32.531883 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:32.532426 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:32.532429 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:32.532431 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:32.532518 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:32.532933 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:32.532937 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:32.532939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:32.533181 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:32.533283 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:32.533701 (+ 418us) server_negotiation.cc:299] Negotiation successful
0808 08:25:32.533801 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 08:25:32.534677 3958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:32.522060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:36595 (local address 127.22.235.194:42463)
0808 08:25:32.522517 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:32.522568 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:32.523458 (+ 890us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:32.524245 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:32.524254 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:32.524674 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:32.525235 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:32.525251 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.526122 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.526126 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:32.526991 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:32.527002 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.527592 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.527596 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:32.527785 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:32.528387 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:32.528407 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:32.530229 (+ 1822us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:32.531997 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:32.532003 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:32.532017 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:32.532335 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:32.532607 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:32.532610 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:32.532636 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:32.532816 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:32.533279 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:32.533284 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:32.533495 (+ 211us) client_negotiation.cc:769] Sending connection context
0808 08:25:32.533640 (+ 145us) client_negotiation.cc:240] Negotiation successful
0808 08:25:32.533869 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":98,"threads_started":1}
I20260808 08:25:32.535853 4151 heartbeater.cc:347] Connected to a master server at 127.22.235.254:36595
I20260808 08:25:32.536005 4151 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:32.536518 4151 heartbeater.cc:511] Master 127.22.235.254:36595 requested a full tablet report, sending...
I20260808 08:25:32.537560 3544 ts_manager.cc:194] Registered new tserver with Master: 417a70ae534e48e580b4dad4ae39dde6 (127.22.235.194:34595)
I20260808 08:25:32.538278 3544 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.235.194@KRBTEST.COM'} at 127.22.235.194:42463
I20260808 08:25:32.538980 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 3919
I20260808 08:25:32.539098 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance
I20260808 08:25:32.539366 23471 external_mini_cluster.cc:1428] Setting key d1934995f995123466653e9df2636a38
I20260808 08:25:32.545089 4160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:32.537994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:36595 (local address 127.22.235.194:44679)
0808 08:25:32.538601 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:32.538613 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:32.538701 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:32.538908 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:32.538910 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:32.539105 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:32.539218 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:32.539225 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.539920 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.539923 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:32.540761 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:32.540772 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.541286 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.541288 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:32.541354 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:32.541764 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:32.541774 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:32.542072 (+ 298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:32.543696 (+ 1624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:32.543700 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:32.543702 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:32.543926 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:32.544160 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:32.544164 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:32.544165 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:32.544259 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:32.544678 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:32.544681 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:32.544760 (+ 79us) client_negotiation.cc:769] Sending connection context
0808 08:25:32.544840 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 08:25:32.544939 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":99,"threads_started":1}
I20260808 08:25:32.545163 3627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:32.538072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:44679 (local address 127.22.235.254:36595)
0808 08:25:32.538183 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:32.538187 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:32.538683 (+ 496us) server_negotiation.cc:407] Connection header received
0808 08:25:32.538778 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:32.538781 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:32.538822 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:32.538882 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:32.539318 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.539807 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.540889 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.541181 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.541289 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:32.542200 (+ 911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:32.542203 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:32.542206 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:32.542232 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:32.543592 (+ 1360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:32.544019 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:32.544021 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:32.544022 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:32.544088 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:32.544335 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:32.544337 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:32.544338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:32.544568 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:32.544691 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:32.544908 (+ 217us) server_negotiation.cc:299] Negotiation successful
0808 08:25:32.545029 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
WARNING: no policy specified for oryx/127.22.235.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.235.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.235.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.195.keytab.
Entry for principal oryx/127.22.235.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.195.keytab.
I20260808 08:25:32.566811 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.195:0
--local_ip_for_outbound_sockets=127.22.235.195
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:36595
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.195.keytab
--principal=oryx/127.22.235.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:38637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:25:32.679263 4164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:32.679505 4164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:32.679553 4164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:32.683557 4164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:32.683609 4164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:32.683652 4164 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 08:25:32.683704 4164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.195
I20260808 08:25:32.688764 4164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:38637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.195.keytab
--principal=oryx/127.22.235.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.235.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.235.254:36595
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.195
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:32.689713 4164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:32.690681 4164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:32.699877 4171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:32.699939 4164 server_base.cc:1034] running on GCE node
W20260808 08:25:32.700047 4175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:32.699882 4172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:32.700516 4164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:32.701155 4164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:32.702309 4164 hybrid_clock.cc:648] HybridClock initialized: now 1786177532702281 us; error 37 us; skew 500 ppm
Aug 08 08:25:32 dist-test-slave-f7th krb5kdc[3469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177532, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.235.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:32.704962 4164 init.cc:377] Logged in from keytab as oryx/127.22.235.195@KRBTEST.COM (short username oryx)
I20260808 08:25:32.705790 4164 webserver.cc:460] Webserver started at http://127.22.235.195:42093/ using document root <none> and password file <none>
I20260808 08:25:32.706353 4164 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:32.706418 4164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:32.706604 4164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:32.708526 4164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-2/data/instance:
uuid: "a89f34ab8de74b31b499851c051728cc"
format_stamp: "Formatted at 2026-08-08 08:25:32 on dist-test-slave-f7th"
server_key: "aba4bd423b8af4bf6c5f762064b62458"
server_key_iv: "be1be12c739f0e2166da39b08ed50269"
server_key_version: "encryptionkey@0"
I20260808 08:25:32.709069 4164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance:
uuid: "a89f34ab8de74b31b499851c051728cc"
format_stamp: "Formatted at 2026-08-08 08:25:32 on dist-test-slave-f7th"
server_key: "aba4bd423b8af4bf6c5f762064b62458"
server_key_iv: "be1be12c739f0e2166da39b08ed50269"
server_key_version: "encryptionkey@0"
I20260808 08:25:32.712543 4164 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 08:25:32.714807 4187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:32.715924 4164 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:25:32.716020 4164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-2/wal
uuid: "a89f34ab8de74b31b499851c051728cc"
format_stamp: "Formatted at 2026-08-08 08:25:32 on dist-test-slave-f7th"
server_key: "aba4bd423b8af4bf6c5f762064b62458"
server_key_iv: "be1be12c739f0e2166da39b08ed50269"
server_key_version: "encryptionkey@0"
I20260808 08:25:32.716110 4164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:32.729696 4164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:32.732162 4164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:32.732318 4164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:32.733736 4164 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:25:32.733798 4164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:32.733860 4164 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:25:32.733887 4164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 08:25:32 dist-test-slave-f7th krb5kdc[3469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177532, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.235.195@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM
I20260808 08:25:32.744341 4164 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.195:45553
I20260808 08:25:32.744386 4393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.195:45553 every 8 connection(s)
I20260808 08:25:32.745373 4164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
I20260808 08:25:32.746831 3627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:32.734505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:34509 (local address 127.22.235.254:36595)
0808 08:25:32.734617 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:32.734620 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:32.735616 (+ 996us) server_negotiation.cc:407] Connection header received
0808 08:25:32.736546 (+ 930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:32.736549 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:32.736638 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:32.736720 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:32.737921 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.738561 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.739810 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.740143 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.740240 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:32.743266 (+ 3026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:32.743271 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:32.743274 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:32.743306 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:32.744769 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:32.745328 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:32.745331 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:32.745332 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:32.745405 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:32.745762 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:32.745765 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:32.745767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:32.746011 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:32.746109 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:32.746594 (+ 485us) server_negotiation.cc:299] Negotiation successful
0808 08:25:32.746695 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 08:25:32.747535 4200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:32.735014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:36595 (local address 127.22.235.195:34509)
0808 08:25:32.735478 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:32.735518 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:32.736340 (+ 822us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:32.736910 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:32.736919 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:32.737328 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:32.737776 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:32.737790 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.738686 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.738689 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:32.739688 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:32.739698 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.740283 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.740287 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:32.740477 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:32.741082 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:32.741097 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:32.743086 (+ 1989us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:32.744902 (+ 1816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:32.744908 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:32.744920 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:32.745240 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:32.745493 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:32.745496 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:32.745498 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:32.745663 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:32.746100 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:32.746105 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:32.746358 (+ 253us) client_negotiation.cc:769] Sending connection context
0808 08:25:32.746536 (+ 178us) client_negotiation.cc:240] Negotiation successful
0808 08:25:32.746767 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":133,"threads_started":1}
I20260808 08:25:32.748600 4394 heartbeater.cc:347] Connected to a master server at 127.22.235.254:36595
I20260808 08:25:32.748770 4394 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:32.749218 4394 heartbeater.cc:511] Master 127.22.235.254:36595 requested a full tablet report, sending...
I20260808 08:25:32.750212 3545 ts_manager.cc:194] Registered new tserver with Master: a89f34ab8de74b31b499851c051728cc (127.22.235.195:45553)
I20260808 08:25:32.750826 3545 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.235.195@KRBTEST.COM'} at 127.22.235.195:34509
I20260808 08:25:32.752779 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 4164
I20260808 08:25:32.752898 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance
I20260808 08:25:32.753173 23471 external_mini_cluster.cc:1428] Setting key 818e976811a0de9546755c0a4e9c0e72
I20260808 08:25:32.755304 23471 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 08:25:32.757640 3627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:32.750749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:34515 (local address 127.22.235.254:36595)
0808 08:25:32.750857 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:32.750860 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:32.751000 (+ 140us) server_negotiation.cc:407] Connection header received
0808 08:25:32.751123 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:32.751126 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:32.751172 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:32.751250 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:32.751667 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.752135 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.753150 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.753439 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.753538 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:32.754620 (+ 1082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:32.754623 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:32.754625 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:32.754657 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:32.756035 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:32.756478 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:32.756480 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:32.756481 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:32.756591 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:32.756929 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:32.756932 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:32.756934 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:32.757109 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:32.757182 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:32.757441 (+ 259us) server_negotiation.cc:299] Negotiation successful
0808 08:25:32.757535 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 08:25:32.757714 4400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:32.750641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:36595 (local address 127.22.235.195:34515)
0808 08:25:32.750915 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:32.750929 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:32.751029 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:32.751263 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:32.751265 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:32.751427 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:32.751550 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:32.751557 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.752252 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.752255 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:32.753043 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:32.753053 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.753554 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.753558 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:32.753649 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:32.754110 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:32.754120 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:32.754504 (+ 384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:32.756134 (+ 1630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:32.756138 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:32.756140 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:32.756394 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:32.756705 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:32.756708 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:32.756709 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:32.756803 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:32.757192 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:32.757195 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:32.757298 (+ 103us) client_negotiation.cc:769] Sending connection context
0808 08:25:32.757405 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 08:25:32.757533 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":100,"threads_started":1}
Aug 08 08:25:32 dist-test-slave-f7th krb5kdc[3469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177532, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 08:25:32 dist-test-slave-f7th krb5kdc[3469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:32 dist-test-slave-f7th krb5kdc[3469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
W20260808 08:25:32.780014 3627 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:32.774316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53848 (local address 127.22.235.254:36595)
0808 08:25:32.774461 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:32.774465 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:32.774477 (+ 12us) server_negotiation.cc:407] Connection header received
0808 08:25:32.774519 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:32.774522 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:32.774560 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:32.774630 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:32.775329 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.775855 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.776696 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.776909 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.777005 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:32.779882 (+ 2877us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53848: BlockingRecv error: recv got EOF from 127.0.0.1:53848 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 08 08:25:32 dist-test-slave-f7th krb5kdc[3469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177532, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM
I20260808 08:25:32.789167 3627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:32.781417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53860 (local address 127.22.235.254:36595)
0808 08:25:32.781544 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:32.781548 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:32.781624 (+ 76us) server_negotiation.cc:407] Connection header received
0808 08:25:32.781750 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:32.781753 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:32.781795 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:32.781855 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:32.782418 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.782932 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.783687 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.783972 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.784062 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:32.786258 (+ 2196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:32.786262 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:32.786265 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:32.786292 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:32.787616 (+ 1324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:32.788022 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:32.788025 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:32.788027 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:32.788105 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:32.788350 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:32.788353 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:32.788355 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:32.788541 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:32.788676 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:32.788903 (+ 227us) server_negotiation.cc:299] Negotiation successful
0808 08:25:32.789015 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 08:25:32.792083 3545 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53860:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 08:25:32.795176 3545 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 08:25:32.809504 4431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:32.803612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55482 (local address 127.22.235.194:34595)
0808 08:25:32.804092 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:32.804105 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:32.804134 (+ 29us) server_negotiation.cc:407] Connection header received
0808 08:25:32.804360 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:32.804366 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:32.804593 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:32.804771 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:25:32.805265 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.806509 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.808076 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.808855 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.809004 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:32.809117 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:32.809276 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 08:25:32.809351 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":108,"threads_started":1}
I20260808 08:25:32.809512 4428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:32.803458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.194:34595 (local address 127.0.0.1:55482)
0808 08:25:32.803968 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:32.804014 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:32.804145 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:32.804973 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:32.804979 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:32.805004 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:25:32.805133 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:32.805140 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.806634 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.806637 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:32.807974 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:32.807984 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.808979 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.808983 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:32.809096 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:32.809133 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 08:25:32.809253 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 08:25:32.809369 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":69,"threads_started":1}
I20260808 08:25:32.809638 4430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:32.804360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.195:45553 (local address 127.0.0.1:36776)
0808 08:25:32.805362 (+ 1002us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:32.805380 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:32.805488 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:32.805893 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:32.805896 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:32.805913 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:25:32.806005 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:32.806011 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.807221 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.807225 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:32.808240 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:32.808248 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.809222 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.809226 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:32.809307 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:32.809328 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 08:25:32.809394 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 08:25:32.809453 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":938,"spinlock_wait_cycles":3200,"thread_start_us":105,"threads_started":1}
I20260808 08:25:32.809883 4436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:32.804916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36776 (local address 127.22.235.195:45553)
0808 08:25:32.805419 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:32.805430 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:32.805450 (+ 20us) server_negotiation.cc:407] Connection header received
0808 08:25:32.805615 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:32.805621 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:32.805780 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:32.805902 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:25:32.806122 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.807091 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.808343 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.808906 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.809038 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:32.809163 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:32.809444 (+ 281us) server_negotiation.cc:299] Negotiation successful
0808 08:25:32.809540 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":168,"threads_started":1}
I20260808 08:25:32.810200 4434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:32.803565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:46367 (local address 127.0.0.1:36110)
0808 08:25:32.804607 (+ 1042us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:32.804648 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:32.804739 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:32.805433 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:32.805436 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:32.805480 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:25:32.805569 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:32.805576 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.806849 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.806853 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:32.807891 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:32.807901 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.809818 (+ 1917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.809821 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:32.809908 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:32.809933 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 08:25:32.810011 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 08:25:32.810075 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":979,"spinlock_wait_cycles":27648,"thread_start_us":92,"threads_started":1}
I20260808 08:25:32.810595 4433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:32.804240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36110 (local address 127.22.235.193:46367)
0808 08:25:32.804505 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:32.804511 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:32.804822 (+ 311us) server_negotiation.cc:407] Connection header received
0808 08:25:32.804902 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:32.804907 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:32.805146 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:32.805280 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:25:32.805689 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.806744 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.808011 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.808855 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.808981 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:32.809117 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:32.810368 (+ 1251us) server_negotiation.cc:299] Negotiation successful
0808 08:25:32.810464 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":77,"threads_started":1}
I20260808 08:25:32.812525 4272 tablet_service.cc:1467] Processing CreateTablet for tablet 8c4c2489f3f9406498319b67e214a01b (DEFAULT_TABLE table=test-table [id=e5cc4ec143324a9899099da9f1dd9541]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:25:32.812677 4026 tablet_service.cc:1467] Processing CreateTablet for tablet 8c4c2489f3f9406498319b67e214a01b (DEFAULT_TABLE table=test-table [id=e5cc4ec143324a9899099da9f1dd9541]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:25:32.812945 3784 tablet_service.cc:1467] Processing CreateTablet for tablet 8c4c2489f3f9406498319b67e214a01b (DEFAULT_TABLE table=test-table [id=e5cc4ec143324a9899099da9f1dd9541]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:25:32.813663 4272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c4c2489f3f9406498319b67e214a01b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:32.813740 4026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c4c2489f3f9406498319b67e214a01b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:32.813792 3784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c4c2489f3f9406498319b67e214a01b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:32.820199 4437 tablet_bootstrap.cc:492] T 8c4c2489f3f9406498319b67e214a01b P a89f34ab8de74b31b499851c051728cc: Bootstrap starting.
I20260808 08:25:32.821839 4439 tablet_bootstrap.cc:492] T 8c4c2489f3f9406498319b67e214a01b P 417a70ae534e48e580b4dad4ae39dde6: Bootstrap starting.
I20260808 08:25:32.821839 4438 tablet_bootstrap.cc:492] T 8c4c2489f3f9406498319b67e214a01b P bc0b21918b524c05aeada9beda0f7b75: Bootstrap starting.
I20260808 08:25:32.822755 4437 tablet_bootstrap.cc:654] T 8c4c2489f3f9406498319b67e214a01b P a89f34ab8de74b31b499851c051728cc: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:32.823450 4437 log.cc:824] T 8c4c2489f3f9406498319b67e214a01b P a89f34ab8de74b31b499851c051728cc: Log is configured to *not* fsync() on all Append() calls
I20260808 08:25:32.823968 4439 tablet_bootstrap.cc:654] T 8c4c2489f3f9406498319b67e214a01b P 417a70ae534e48e580b4dad4ae39dde6: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:32.823977 4438 tablet_bootstrap.cc:654] T 8c4c2489f3f9406498319b67e214a01b P bc0b21918b524c05aeada9beda0f7b75: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:32.824847 4439 log.cc:824] T 8c4c2489f3f9406498319b67e214a01b P 417a70ae534e48e580b4dad4ae39dde6: Log is configured to *not* fsync() on all Append() calls
I20260808 08:25:32.824847 4438 log.cc:824] T 8c4c2489f3f9406498319b67e214a01b P bc0b21918b524c05aeada9beda0f7b75: Log is configured to *not* fsync() on all Append() calls
I20260808 08:25:32.825273 4437 tablet_bootstrap.cc:492] T 8c4c2489f3f9406498319b67e214a01b P a89f34ab8de74b31b499851c051728cc: No bootstrap required, opened a new log
I20260808 08:25:32.825476 4437 ts_tablet_manager.cc:1397] T 8c4c2489f3f9406498319b67e214a01b P a89f34ab8de74b31b499851c051728cc: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 08:25:32.826725 4439 tablet_bootstrap.cc:492] T 8c4c2489f3f9406498319b67e214a01b P 417a70ae534e48e580b4dad4ae39dde6: No bootstrap required, opened a new log
I20260808 08:25:32.826725 4438 tablet_bootstrap.cc:492] T 8c4c2489f3f9406498319b67e214a01b P bc0b21918b524c05aeada9beda0f7b75: No bootstrap required, opened a new log
I20260808 08:25:32.826936 4439 ts_tablet_manager.cc:1397] T 8c4c2489f3f9406498319b67e214a01b P 417a70ae534e48e580b4dad4ae39dde6: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260808 08:25:32.826936 4438 ts_tablet_manager.cc:1397] T 8c4c2489f3f9406498319b67e214a01b P bc0b21918b524c05aeada9beda0f7b75: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260808 08:25:32.828680 4437 raft_consensus.cc:348] T 8c4c2489f3f9406498319b67e214a01b P a89f34ab8de74b31b499851c051728cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } }
I20260808 08:25:32.828912 4437 raft_consensus.cc:374] T 8c4c2489f3f9406498319b67e214a01b P a89f34ab8de74b31b499851c051728cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:32.828963 4437 raft_consensus.cc:729] T 8c4c2489f3f9406498319b67e214a01b P a89f34ab8de74b31b499851c051728cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a89f34ab8de74b31b499851c051728cc, State: Initialized, Role: FOLLOWER
I20260808 08:25:32.829368 4437 consensus_queue.cc:260] T 8c4c2489f3f9406498319b67e214a01b P a89f34ab8de74b31b499851c051728cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } }
I20260808 08:25:32.830021 4394 heartbeater.cc:503] Master 127.22.235.254:36595 was elected leader, sending a full tablet report...
I20260808 08:25:32.830277 4437 ts_tablet_manager.cc:1428] T 8c4c2489f3f9406498319b67e214a01b P a89f34ab8de74b31b499851c051728cc: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 08:25:32.831020 4438 raft_consensus.cc:348] T 8c4c2489f3f9406498319b67e214a01b P bc0b21918b524c05aeada9beda0f7b75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } }
I20260808 08:25:32.831281 4438 raft_consensus.cc:374] T 8c4c2489f3f9406498319b67e214a01b P bc0b21918b524c05aeada9beda0f7b75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:32.831334 4438 raft_consensus.cc:729] T 8c4c2489f3f9406498319b67e214a01b P bc0b21918b524c05aeada9beda0f7b75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc0b21918b524c05aeada9beda0f7b75, State: Initialized, Role: FOLLOWER
I20260808 08:25:32.831755 4438 consensus_queue.cc:260] T 8c4c2489f3f9406498319b67e214a01b P bc0b21918b524c05aeada9beda0f7b75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } }
I20260808 08:25:32.832000 4439 raft_consensus.cc:348] T 8c4c2489f3f9406498319b67e214a01b P 417a70ae534e48e580b4dad4ae39dde6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } }
I20260808 08:25:32.832249 4439 raft_consensus.cc:374] T 8c4c2489f3f9406498319b67e214a01b P 417a70ae534e48e580b4dad4ae39dde6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:32.832307 4439 raft_consensus.cc:729] T 8c4c2489f3f9406498319b67e214a01b P 417a70ae534e48e580b4dad4ae39dde6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 417a70ae534e48e580b4dad4ae39dde6, State: Initialized, Role: FOLLOWER
I20260808 08:25:32.832556 3909 heartbeater.cc:503] Master 127.22.235.254:36595 was elected leader, sending a full tablet report...
I20260808 08:25:32.832738 4439 consensus_queue.cc:260] T 8c4c2489f3f9406498319b67e214a01b P 417a70ae534e48e580b4dad4ae39dde6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } }
I20260808 08:25:32.833536 4151 heartbeater.cc:503] Master 127.22.235.254:36595 was elected leader, sending a full tablet report...
I20260808 08:25:32.833730 4439 ts_tablet_manager.cc:1428] T 8c4c2489f3f9406498319b67e214a01b P 417a70ae534e48e580b4dad4ae39dde6: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260808 08:25:32.834347 4438 ts_tablet_manager.cc:1428] T 8c4c2489f3f9406498319b67e214a01b P bc0b21918b524c05aeada9beda0f7b75: Time spent starting tablet: real 0.007s user 0.005s sys 0.002s
I20260808 08:25:32.871637 4447 raft_consensus.cc:482] T 8c4c2489f3f9406498319b67e214a01b P a89f34ab8de74b31b499851c051728cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 08:25:32.871858 4447 raft_consensus.cc:504] T 8c4c2489f3f9406498319b67e214a01b P a89f34ab8de74b31b499851c051728cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } }
I20260808 08:25:32.873113 4447 leader_election.cc:288] T 8c4c2489f3f9406498319b67e214a01b P a89f34ab8de74b31b499851c051728cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 417a70ae534e48e580b4dad4ae39dde6 (127.22.235.194:34595), bc0b21918b524c05aeada9beda0f7b75 (127.22.235.193:46367)
I20260808 08:25:32.876507 4200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:32.873333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.194:34595 (local address 127.22.235.195:52421)
0808 08:25:32.873490 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:32.873506 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:32.873601 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:32.873886 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:32.873889 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:32.873909 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:25:32.874019 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:32.874025 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.874733 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.874736 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:32.875508 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:32.875516 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.876113 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.876117 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:32.876222 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:32.876240 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 08:25:32.876304 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 08:25:32.876373 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 08:25:32.876513 4431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:32.873510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:52421 (local address 127.22.235.194:34595)
0808 08:25:32.873632 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:32.873637 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:32.873653 (+ 16us) server_negotiation.cc:407] Connection header received
0808 08:25:32.873709 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:32.873712 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:32.873761 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:32.873837 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:25:32.874122 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.874632 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.875627 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.876011 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.876113 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:32.876170 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:32.876327 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 08:25:32.876393 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 08:25:32.877158 4433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:32.873515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:58135 (local address 127.22.235.193:46367)
0808 08:25:32.873652 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:32.873657 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:32.874001 (+ 344us) server_negotiation.cc:407] Connection header received
0808 08:25:32.874117 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:32.874120 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:32.874161 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:32.874222 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:25:32.874782 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.875254 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.876208 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.876602 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.876737 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:32.876780 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:32.876984 (+ 204us) server_negotiation.cc:299] Negotiation successful
0808 08:25:32.877051 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 08:25:32.877173 4450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:32.873333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:46367 (local address 127.22.235.195:58135)
0808 08:25:32.873905 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:32.873916 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:32.874020 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:32.874238 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:32.874241 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:32.874256 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:25:32.874661 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:32.874668 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.875354 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.875358 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:32.876087 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:32.876096 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.876775 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.876779 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:32.876871 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:32.876892 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 08:25:32.876968 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 08:25:32.877037 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"mutex_wait_us":228,"thread_start_us":143,"threads_started":1}
I20260808 08:25:32.877182 4075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c4c2489f3f9406498319b67e214a01b" candidate_uuid: "a89f34ab8de74b31b499851c051728cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "417a70ae534e48e580b4dad4ae39dde6" is_pre_election: true
I20260808 08:25:32.877506 4075 raft_consensus.cc:2454] T 8c4c2489f3f9406498319b67e214a01b P 417a70ae534e48e580b4dad4ae39dde6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a89f34ab8de74b31b499851c051728cc in term 0.
I20260808 08:25:32.877663 3818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c4c2489f3f9406498319b67e214a01b" candidate_uuid: "a89f34ab8de74b31b499851c051728cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc0b21918b524c05aeada9beda0f7b75" is_pre_election: true
I20260808 08:25:32.877952 3818 raft_consensus.cc:2454] T 8c4c2489f3f9406498319b67e214a01b P bc0b21918b524c05aeada9beda0f7b75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a89f34ab8de74b31b499851c051728cc in term 0.
I20260808 08:25:32.878010 4193 leader_election.cc:302] T 8c4c2489f3f9406498319b67e214a01b P a89f34ab8de74b31b499851c051728cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 417a70ae534e48e580b4dad4ae39dde6, a89f34ab8de74b31b499851c051728cc; no voters:
I20260808 08:25:32.878283 4447 raft_consensus.cc:2781] T 8c4c2489f3f9406498319b67e214a01b P a89f34ab8de74b31b499851c051728cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 08:25:32.878360 4447 raft_consensus.cc:482] T 8c4c2489f3f9406498319b67e214a01b P a89f34ab8de74b31b499851c051728cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 08:25:32.878391 4447 raft_consensus.cc:3037] T 8c4c2489f3f9406498319b67e214a01b P a89f34ab8de74b31b499851c051728cc [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:32.879401 4447 raft_consensus.cc:504] T 8c4c2489f3f9406498319b67e214a01b P a89f34ab8de74b31b499851c051728cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } }
I20260808 08:25:32.879731 4447 leader_election.cc:288] T 8c4c2489f3f9406498319b67e214a01b P a89f34ab8de74b31b499851c051728cc [CANDIDATE]: Term 1 election: Requested vote from peers 417a70ae534e48e580b4dad4ae39dde6 (127.22.235.194:34595), bc0b21918b524c05aeada9beda0f7b75 (127.22.235.193:46367)
I20260808 08:25:32.880043 3818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c4c2489f3f9406498319b67e214a01b" candidate_uuid: "a89f34ab8de74b31b499851c051728cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc0b21918b524c05aeada9beda0f7b75"
I20260808 08:25:32.880168 3818 raft_consensus.cc:3037] T 8c4c2489f3f9406498319b67e214a01b P bc0b21918b524c05aeada9beda0f7b75 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:32.881176 3818 raft_consensus.cc:2454] T 8c4c2489f3f9406498319b67e214a01b P bc0b21918b524c05aeada9beda0f7b75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a89f34ab8de74b31b499851c051728cc in term 1.
I20260808 08:25:32.881433 4075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c4c2489f3f9406498319b67e214a01b" candidate_uuid: "a89f34ab8de74b31b499851c051728cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "417a70ae534e48e580b4dad4ae39dde6"
I20260808 08:25:32.881466 4190 leader_election.cc:302] T 8c4c2489f3f9406498319b67e214a01b P a89f34ab8de74b31b499851c051728cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a89f34ab8de74b31b499851c051728cc, bc0b21918b524c05aeada9beda0f7b75; no voters:
I20260808 08:25:32.881572 4075 raft_consensus.cc:3037] T 8c4c2489f3f9406498319b67e214a01b P 417a70ae534e48e580b4dad4ae39dde6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:32.881654 4447 raft_consensus.cc:2781] T 8c4c2489f3f9406498319b67e214a01b P a89f34ab8de74b31b499851c051728cc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:25:32.881875 4447 raft_consensus.cc:686] T 8c4c2489f3f9406498319b67e214a01b P a89f34ab8de74b31b499851c051728cc [term 1 LEADER]: Becoming Leader. State: Replica: a89f34ab8de74b31b499851c051728cc, State: Running, Role: LEADER
I20260808 08:25:32.882230 4447 consensus_queue.cc:237] T 8c4c2489f3f9406498319b67e214a01b P a89f34ab8de74b31b499851c051728cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } }
I20260808 08:25:32.882856 4075 raft_consensus.cc:2454] T 8c4c2489f3f9406498319b67e214a01b P 417a70ae534e48e580b4dad4ae39dde6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a89f34ab8de74b31b499851c051728cc in term 1.
I20260808 08:25:32.885633 3542 catalog_manager.cc:5495] T 8c4c2489f3f9406498319b67e214a01b P a89f34ab8de74b31b499851c051728cc reported cstate change: term changed from 0 to 1, leader changed from <none> to a89f34ab8de74b31b499851c051728cc (127.22.235.195). New cstate: current_term: 1 leader_uuid: "a89f34ab8de74b31b499851c051728cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } health_report { overall_health: HEALTHY } } }
I20260808 08:25:32.909158 4436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:32.906033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36778 (local address 127.22.235.195:45553)
0808 08:25:32.906145 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:32.906148 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:32.906161 (+ 13us) server_negotiation.cc:407] Connection header received
0808 08:25:32.906276 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:32.906279 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:32.906353 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:32.906442 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:25:32.906636 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.907169 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.907937 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:32.908149 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:32.908223 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:32.908412 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:25:32.908893 (+ 481us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 08:25:32.908975 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:32.909003 (+ 28us) server_negotiation.cc:299] Negotiation successful
0808 08:25:32.909052 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 08:25:32.914911 3818 raft_consensus.cc:1261] T 8c4c2489f3f9406498319b67e214a01b P bc0b21918b524c05aeada9beda0f7b75 [term 1 FOLLOWER]: Refusing update from remote peer a89f34ab8de74b31b499851c051728cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:25:32.915199 4075 raft_consensus.cc:1261] T 8c4c2489f3f9406498319b67e214a01b P 417a70ae534e48e580b4dad4ae39dde6 [term 1 FOLLOWER]: Refusing update from remote peer a89f34ab8de74b31b499851c051728cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:25:32.915597 4447 consensus_queue.cc:1035] T 8c4c2489f3f9406498319b67e214a01b P a89f34ab8de74b31b499851c051728cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:25:32.915777 4451 consensus_queue.cc:1035] T 8c4c2489f3f9406498319b67e214a01b P a89f34ab8de74b31b499851c051728cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:25:32.923024 4456 mvcc.cc:205] Tried to move back new op lower bound from 7316183174812692480 to 7316183174688899072. Current Snapshot: MvccSnapshot[applied={T|T < 7316183174812692480}]
I20260808 08:25:32.925001 4459 mvcc.cc:205] Tried to move back new op lower bound from 7316183174812692480 to 7316183174688899072. Current Snapshot: MvccSnapshot[applied={T|T < 7316183174812692480}]
I20260808 08:25:32.925450 4458 mvcc.cc:205] Tried to move back new op lower bound from 7316183174812692480 to 7316183174688899072. Current Snapshot: MvccSnapshot[applied={T|T < 7316183174812692480}]
I20260808 08:25:32.932176 3544 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:25:32.932332 3544 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:25:32.934688 3544 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a626e0cf38834d9992e58716fe7ae506: Sending DeleteTablet for 3 replicas of tablet 8c4c2489f3f9406498319b67e214a01b
I20260808 08:25:32.935487 3784 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c4c2489f3f9406498319b67e214a01b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:25:32 UTC) from {username='oryx', principal='oryx/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:36110
I20260808 08:25:32.935531 4272 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c4c2489f3f9406498319b67e214a01b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:25:32 UTC) from {username='oryx', principal='oryx/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:36776
I20260808 08:25:32.935570 4026 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c4c2489f3f9406498319b67e214a01b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:25:32 UTC) from {username='oryx', principal='oryx/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:55482
I20260808 08:25:32.935978 4473 tablet_replica.cc:330] T 8c4c2489f3f9406498319b67e214a01b P a89f34ab8de74b31b499851c051728cc: stopping tablet replica
I20260808 08:25:32.936281 4473 raft_consensus.cc:2229] T 8c4c2489f3f9406498319b67e214a01b P a89f34ab8de74b31b499851c051728cc [term 1 LEADER]: Raft consensus shutting down.
I20260808 08:25:32.936327 4475 tablet_replica.cc:330] T 8c4c2489f3f9406498319b67e214a01b P 417a70ae534e48e580b4dad4ae39dde6: stopping tablet replica
I20260808 08:25:32.936635 4475 raft_consensus.cc:2229] T 8c4c2489f3f9406498319b67e214a01b P 417a70ae534e48e580b4dad4ae39dde6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 08:25:32.936800 4473 raft_consensus.cc:2258] T 8c4c2489f3f9406498319b67e214a01b P a89f34ab8de74b31b499851c051728cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:25:32.937172 4471 tablet_replica.cc:330] T 8c4c2489f3f9406498319b67e214a01b P bc0b21918b524c05aeada9beda0f7b75: stopping tablet replica
I20260808 08:25:32.937101 3544 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53860:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 08:25:32.937438 4471 raft_consensus.cc:2229] T 8c4c2489f3f9406498319b67e214a01b P bc0b21918b524c05aeada9beda0f7b75 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260808 08:25:32.937520 3544 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 08:25:32.937705 4471 raft_consensus.cc:2258] T 8c4c2489f3f9406498319b67e214a01b P bc0b21918b524c05aeada9beda0f7b75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:25:32.938235 4475 raft_consensus.cc:2258] T 8c4c2489f3f9406498319b67e214a01b P 417a70ae534e48e580b4dad4ae39dde6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:25:32.939743 4475 ts_tablet_manager.cc:1905] T 8c4c2489f3f9406498319b67e214a01b P 417a70ae534e48e580b4dad4ae39dde6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:25:32.940937 4473 ts_tablet_manager.cc:1905] T 8c4c2489f3f9406498319b67e214a01b P a89f34ab8de74b31b499851c051728cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:25:32.941959 3784 tablet_service.cc:1467] Processing CreateTablet for tablet 2338a2e248b34eb495819568927ed019 (DEFAULT_TABLE table=test-table [id=44bdf8fa616445d8a3eda472490a007a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:25:32.942248 3784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2338a2e248b34eb495819568927ed019. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:32.942625 4475 ts_tablet_manager.cc:1918] T 8c4c2489f3f9406498319b67e214a01b P 417a70ae534e48e580b4dad4ae39dde6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 08:25:32.942723 4475 log.cc:1196] T 8c4c2489f3f9406498319b67e214a01b P 417a70ae534e48e580b4dad4ae39dde6: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-1/wal/wals/8c4c2489f3f9406498319b67e214a01b
I20260808 08:25:32.943029 4475 ts_tablet_manager.cc:1939] T 8c4c2489f3f9406498319b67e214a01b P 417a70ae534e48e580b4dad4ae39dde6: Deleting consensus metadata
I20260808 08:25:32.944216 4438 tablet_bootstrap.cc:492] T 2338a2e248b34eb495819568927ed019 P bc0b21918b524c05aeada9beda0f7b75: Bootstrap starting.
I20260808 08:25:32.944423 3524 catalog_manager.cc:4841] TS 417a70ae534e48e580b4dad4ae39dde6 (127.22.235.194:34595): tablet 8c4c2489f3f9406498319b67e214a01b (table test-table [id=e5cc4ec143324a9899099da9f1dd9541]) successfully deleted
I20260808 08:25:32.944959 4026 tablet_service.cc:1467] Processing CreateTablet for tablet 2338a2e248b34eb495819568927ed019 (DEFAULT_TABLE table=test-table [id=44bdf8fa616445d8a3eda472490a007a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:25:32.945210 4438 tablet_bootstrap.cc:654] T 2338a2e248b34eb495819568927ed019 P bc0b21918b524c05aeada9beda0f7b75: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:32.945221 4026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2338a2e248b34eb495819568927ed019. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:32.945276 4272 tablet_service.cc:1467] Processing CreateTablet for tablet 2338a2e248b34eb495819568927ed019 (DEFAULT_TABLE table=test-table [id=44bdf8fa616445d8a3eda472490a007a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:25:32.945514 4272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2338a2e248b34eb495819568927ed019. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:32.946352 4438 tablet_bootstrap.cc:492] T 2338a2e248b34eb495819568927ed019 P bc0b21918b524c05aeada9beda0f7b75: No bootstrap required, opened a new log
I20260808 08:25:32.946425 4438 ts_tablet_manager.cc:1397] T 2338a2e248b34eb495819568927ed019 P bc0b21918b524c05aeada9beda0f7b75: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 08:25:32.946863 4438 raft_consensus.cc:348] T 2338a2e248b34eb495819568927ed019 P bc0b21918b524c05aeada9beda0f7b75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } }
I20260808 08:25:32.946976 4438 raft_consensus.cc:374] T 2338a2e248b34eb495819568927ed019 P bc0b21918b524c05aeada9beda0f7b75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:32.947008 4438 raft_consensus.cc:729] T 2338a2e248b34eb495819568927ed019 P bc0b21918b524c05aeada9beda0f7b75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc0b21918b524c05aeada9beda0f7b75, State: Initialized, Role: FOLLOWER
I20260808 08:25:32.947124 4438 consensus_queue.cc:260] T 2338a2e248b34eb495819568927ed019 P bc0b21918b524c05aeada9beda0f7b75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } }
I20260808 08:25:32.947295 4439 tablet_bootstrap.cc:492] T 2338a2e248b34eb495819568927ed019 P 417a70ae534e48e580b4dad4ae39dde6: Bootstrap starting.
I20260808 08:25:32.947358 4438 ts_tablet_manager.cc:1428] T 2338a2e248b34eb495819568927ed019 P bc0b21918b524c05aeada9beda0f7b75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 08:25:32.947757 4437 tablet_bootstrap.cc:492] T 2338a2e248b34eb495819568927ed019 P a89f34ab8de74b31b499851c051728cc: Bootstrap starting.
I20260808 08:25:32.948199 4439 tablet_bootstrap.cc:654] T 2338a2e248b34eb495819568927ed019 P 417a70ae534e48e580b4dad4ae39dde6: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:32.948694 4437 tablet_bootstrap.cc:654] T 2338a2e248b34eb495819568927ed019 P a89f34ab8de74b31b499851c051728cc: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:32.949884 4439 tablet_bootstrap.cc:492] T 2338a2e248b34eb495819568927ed019 P 417a70ae534e48e580b4dad4ae39dde6: No bootstrap required, opened a new log
I20260808 08:25:32.949965 4439 ts_tablet_manager.cc:1397] T 2338a2e248b34eb495819568927ed019 P 417a70ae534e48e580b4dad4ae39dde6: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 08:25:32.950121 4437 tablet_bootstrap.cc:492] T 2338a2e248b34eb495819568927ed019 P a89f34ab8de74b31b499851c051728cc: No bootstrap required, opened a new log
I20260808 08:25:32.950194 4437 ts_tablet_manager.cc:1397] T 2338a2e248b34eb495819568927ed019 P a89f34ab8de74b31b499851c051728cc: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 08:25:32.950438 4473 ts_tablet_manager.cc:1918] T 8c4c2489f3f9406498319b67e214a01b P a89f34ab8de74b31b499851c051728cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 08:25:32.950397 4439 raft_consensus.cc:348] T 2338a2e248b34eb495819568927ed019 P 417a70ae534e48e580b4dad4ae39dde6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } }
I20260808 08:25:32.950503 4439 raft_consensus.cc:374] T 2338a2e248b34eb495819568927ed019 P 417a70ae534e48e580b4dad4ae39dde6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:32.950536 4473 log.cc:1196] T 8c4c2489f3f9406498319b67e214a01b P a89f34ab8de74b31b499851c051728cc: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-2/wal/wals/8c4c2489f3f9406498319b67e214a01b
I20260808 08:25:32.950536 4439 raft_consensus.cc:729] T 2338a2e248b34eb495819568927ed019 P 417a70ae534e48e580b4dad4ae39dde6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 417a70ae534e48e580b4dad4ae39dde6, State: Initialized, Role: FOLLOWER
I20260808 08:25:32.950598 4437 raft_consensus.cc:348] T 2338a2e248b34eb495819568927ed019 P a89f34ab8de74b31b499851c051728cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } }
I20260808 08:25:32.950691 4437 raft_consensus.cc:374] T 2338a2e248b34eb495819568927ed019 P a89f34ab8de74b31b499851c051728cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:32.950659 4439 consensus_queue.cc:260] T 2338a2e248b34eb495819568927ed019 P 417a70ae534e48e580b4dad4ae39dde6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } }
I20260808 08:25:32.950719 4437 raft_consensus.cc:729] T 2338a2e248b34eb495819568927ed019 P a89f34ab8de74b31b499851c051728cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a89f34ab8de74b31b499851c051728cc, State: Initialized, Role: FOLLOWER
I20260808 08:25:32.950848 4473 ts_tablet_manager.cc:1939] T 8c4c2489f3f9406498319b67e214a01b P a89f34ab8de74b31b499851c051728cc: Deleting consensus metadata
I20260808 08:25:32.950835 4437 consensus_queue.cc:260] T 2338a2e248b34eb495819568927ed019 P a89f34ab8de74b31b499851c051728cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } }
I20260808 08:25:32.950920 4439 ts_tablet_manager.cc:1428] T 2338a2e248b34eb495819568927ed019 P 417a70ae534e48e580b4dad4ae39dde6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 08:25:32.951076 4437 ts_tablet_manager.cc:1428] T 2338a2e248b34eb495819568927ed019 P a89f34ab8de74b31b499851c051728cc: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 08:25:32.952325 3520 catalog_manager.cc:4841] TS a89f34ab8de74b31b499851c051728cc (127.22.235.195:45553): tablet 8c4c2489f3f9406498319b67e214a01b (table test-table [id=e5cc4ec143324a9899099da9f1dd9541]) successfully deleted
I20260808 08:25:32.953208 4471 ts_tablet_manager.cc:1905] T 8c4c2489f3f9406498319b67e214a01b P bc0b21918b524c05aeada9beda0f7b75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:25:32.955740 4471 ts_tablet_manager.cc:1918] T 8c4c2489f3f9406498319b67e214a01b P bc0b21918b524c05aeada9beda0f7b75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 08:25:32.955816 4471 log.cc:1196] T 8c4c2489f3f9406498319b67e214a01b P bc0b21918b524c05aeada9beda0f7b75: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-0/wal/wals/8c4c2489f3f9406498319b67e214a01b
I20260808 08:25:32.956099 4471 ts_tablet_manager.cc:1939] T 8c4c2489f3f9406498319b67e214a01b P bc0b21918b524c05aeada9beda0f7b75: Deleting consensus metadata
I20260808 08:25:32.957275 3522 catalog_manager.cc:4841] TS bc0b21918b524c05aeada9beda0f7b75 (127.22.235.193:46367): tablet 8c4c2489f3f9406498319b67e214a01b (table test-table [id=e5cc4ec143324a9899099da9f1dd9541]) successfully deleted
W20260808 08:25:32.996238 4395 tablet.cc:2369] T 2338a2e248b34eb495819568927ed019 P a89f34ab8de74b31b499851c051728cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 08:25:33.033443 4152 tablet.cc:2369] T 2338a2e248b34eb495819568927ed019 P 417a70ae534e48e580b4dad4ae39dde6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 08:25:33.050565 3910 tablet.cc:2369] T 2338a2e248b34eb495819568927ed019 P bc0b21918b524c05aeada9beda0f7b75: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 08:25:33.143960 4464 raft_consensus.cc:482] T 2338a2e248b34eb495819568927ed019 P a89f34ab8de74b31b499851c051728cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 08:25:33.144134 4464 raft_consensus.cc:504] T 2338a2e248b34eb495819568927ed019 P a89f34ab8de74b31b499851c051728cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } }
I20260808 08:25:33.144541 4464 leader_election.cc:288] T 2338a2e248b34eb495819568927ed019 P a89f34ab8de74b31b499851c051728cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc0b21918b524c05aeada9beda0f7b75 (127.22.235.193:46367), 417a70ae534e48e580b4dad4ae39dde6 (127.22.235.194:34595)
I20260808 08:25:33.145025 3818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2338a2e248b34eb495819568927ed019" candidate_uuid: "a89f34ab8de74b31b499851c051728cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc0b21918b524c05aeada9beda0f7b75" is_pre_election: true
I20260808 08:25:33.145030 4075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2338a2e248b34eb495819568927ed019" candidate_uuid: "a89f34ab8de74b31b499851c051728cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "417a70ae534e48e580b4dad4ae39dde6" is_pre_election: true
I20260808 08:25:33.145172 3818 raft_consensus.cc:2454] T 2338a2e248b34eb495819568927ed019 P bc0b21918b524c05aeada9beda0f7b75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a89f34ab8de74b31b499851c051728cc in term 0.
I20260808 08:25:33.145177 4075 raft_consensus.cc:2454] T 2338a2e248b34eb495819568927ed019 P 417a70ae534e48e580b4dad4ae39dde6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a89f34ab8de74b31b499851c051728cc in term 0.
I20260808 08:25:33.145454 4190 leader_election.cc:302] T 2338a2e248b34eb495819568927ed019 P a89f34ab8de74b31b499851c051728cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a89f34ab8de74b31b499851c051728cc, bc0b21918b524c05aeada9beda0f7b75; no voters:
I20260808 08:25:33.145638 4464 raft_consensus.cc:2781] T 2338a2e248b34eb495819568927ed019 P a89f34ab8de74b31b499851c051728cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 08:25:33.145699 4464 raft_consensus.cc:482] T 2338a2e248b34eb495819568927ed019 P a89f34ab8de74b31b499851c051728cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 08:25:33.145725 4464 raft_consensus.cc:3037] T 2338a2e248b34eb495819568927ed019 P a89f34ab8de74b31b499851c051728cc [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:33.146724 4464 raft_consensus.cc:504] T 2338a2e248b34eb495819568927ed019 P a89f34ab8de74b31b499851c051728cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } }
I20260808 08:25:33.147065 4464 leader_election.cc:288] T 2338a2e248b34eb495819568927ed019 P a89f34ab8de74b31b499851c051728cc [CANDIDATE]: Term 1 election: Requested vote from peers bc0b21918b524c05aeada9beda0f7b75 (127.22.235.193:46367), 417a70ae534e48e580b4dad4ae39dde6 (127.22.235.194:34595)
I20260808 08:25:33.147420 4075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2338a2e248b34eb495819568927ed019" candidate_uuid: "a89f34ab8de74b31b499851c051728cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "417a70ae534e48e580b4dad4ae39dde6"
I20260808 08:25:33.147449 3818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2338a2e248b34eb495819568927ed019" candidate_uuid: "a89f34ab8de74b31b499851c051728cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc0b21918b524c05aeada9beda0f7b75"
I20260808 08:25:33.147553 3818 raft_consensus.cc:3037] T 2338a2e248b34eb495819568927ed019 P bc0b21918b524c05aeada9beda0f7b75 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:33.147555 4075 raft_consensus.cc:3037] T 2338a2e248b34eb495819568927ed019 P 417a70ae534e48e580b4dad4ae39dde6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:33.148495 4075 raft_consensus.cc:2454] T 2338a2e248b34eb495819568927ed019 P 417a70ae534e48e580b4dad4ae39dde6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a89f34ab8de74b31b499851c051728cc in term 1.
I20260808 08:25:33.148489 3818 raft_consensus.cc:2454] T 2338a2e248b34eb495819568927ed019 P bc0b21918b524c05aeada9beda0f7b75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a89f34ab8de74b31b499851c051728cc in term 1.
I20260808 08:25:33.148798 4190 leader_election.cc:302] T 2338a2e248b34eb495819568927ed019 P a89f34ab8de74b31b499851c051728cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a89f34ab8de74b31b499851c051728cc, bc0b21918b524c05aeada9beda0f7b75; no voters:
I20260808 08:25:33.148993 4464 raft_consensus.cc:2781] T 2338a2e248b34eb495819568927ed019 P a89f34ab8de74b31b499851c051728cc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:25:33.149087 4464 raft_consensus.cc:686] T 2338a2e248b34eb495819568927ed019 P a89f34ab8de74b31b499851c051728cc [term 1 LEADER]: Becoming Leader. State: Replica: a89f34ab8de74b31b499851c051728cc, State: Running, Role: LEADER
I20260808 08:25:33.149225 4464 consensus_queue.cc:237] T 2338a2e248b34eb495819568927ed019 P a89f34ab8de74b31b499851c051728cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } }
I20260808 08:25:33.150914 3544 catalog_manager.cc:5495] T 2338a2e248b34eb495819568927ed019 P a89f34ab8de74b31b499851c051728cc reported cstate change: term changed from 0 to 1, leader changed from <none> to a89f34ab8de74b31b499851c051728cc (127.22.235.195). New cstate: current_term: 1 leader_uuid: "a89f34ab8de74b31b499851c051728cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } health_report { overall_health: HEALTHY } } }
Aug 08 08:25:33 dist-test-slave-f7th krb5kdc[3469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177531, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.235.254@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM
I20260808 08:25:33.229723 4491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:33.218334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:36595 (local address 127.0.0.1:53872)
0808 08:25:33.218602 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:33.218614 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:33.218719 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:33.218932 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:33.218933 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:33.219387 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:33.219503 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:33.219511 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:33.220245 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:33.220248 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:33.220825 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:33.220834 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:33.221192 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:33.221195 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:33.221250 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:33.225445 (+ 4195us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:33.225458 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:33.226886 (+ 1428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:33.228290 (+ 1404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:33.228299 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:33.228306 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:33.228545 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:33.228786 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:33.228788 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:33.228790 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:33.228867 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:33.229203 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:33.229208 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:33.229314 (+ 106us) client_negotiation.cc:769] Sending connection context
0808 08:25:33.229401 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 08:25:33.229535 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":94,"threads_started":1}
I20260808 08:25:33.229725 3627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:33.218392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53872 (local address 127.22.235.254:36595)
0808 08:25:33.218554 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:33.218559 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:33.218681 (+ 122us) server_negotiation.cc:407] Connection header received
0808 08:25:33.218793 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:33.218795 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:33.218839 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:33.218903 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:33.219598 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:33.220163 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:33.220912 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:33.221115 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:33.221185 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:33.227025 (+ 5840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:33.227029 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:33.227031 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:33.227054 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:33.228193 (+ 1139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:33.228637 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:33.228640 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:33.228642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:33.228717 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:33.228946 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:33.228948 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:33.228949 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:33.229127 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:33.229200 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:33.229442 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 08:25:33.229588 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":63}
I20260808 08:25:33.233543 3544 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:53872:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 08:25:33.234272 3544 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 08:25:33.238940 4026 tablet_service.cc:1467] Processing CreateTablet for tablet 9423faadbcef49c2a66144eee1991476 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c1c8d36a77d04a088386e7fa4c0517f0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 08:25:33.238943 3784 tablet_service.cc:1467] Processing CreateTablet for tablet 9423faadbcef49c2a66144eee1991476 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c1c8d36a77d04a088386e7fa4c0517f0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 08:25:33.239224 4272 tablet_service.cc:1467] Processing CreateTablet for tablet 9423faadbcef49c2a66144eee1991476 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c1c8d36a77d04a088386e7fa4c0517f0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 08:25:33.239360 4026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9423faadbcef49c2a66144eee1991476. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:33.239365 3784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9423faadbcef49c2a66144eee1991476. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:33.239573 4272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9423faadbcef49c2a66144eee1991476. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:33.241798 4439 tablet_bootstrap.cc:492] T 9423faadbcef49c2a66144eee1991476 P 417a70ae534e48e580b4dad4ae39dde6: Bootstrap starting.
I20260808 08:25:33.242179 4437 tablet_bootstrap.cc:492] T 9423faadbcef49c2a66144eee1991476 P a89f34ab8de74b31b499851c051728cc: Bootstrap starting.
I20260808 08:25:33.242197 4438 tablet_bootstrap.cc:492] T 9423faadbcef49c2a66144eee1991476 P bc0b21918b524c05aeada9beda0f7b75: Bootstrap starting.
I20260808 08:25:33.242748 4439 tablet_bootstrap.cc:654] T 9423faadbcef49c2a66144eee1991476 P 417a70ae534e48e580b4dad4ae39dde6: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:33.242983 4437 tablet_bootstrap.cc:654] T 9423faadbcef49c2a66144eee1991476 P a89f34ab8de74b31b499851c051728cc: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:33.243058 4438 tablet_bootstrap.cc:654] T 9423faadbcef49c2a66144eee1991476 P bc0b21918b524c05aeada9beda0f7b75: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:33.243953 4439 tablet_bootstrap.cc:492] T 9423faadbcef49c2a66144eee1991476 P 417a70ae534e48e580b4dad4ae39dde6: No bootstrap required, opened a new log
I20260808 08:25:33.244050 4439 ts_tablet_manager.cc:1397] T 9423faadbcef49c2a66144eee1991476 P 417a70ae534e48e580b4dad4ae39dde6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 08:25:33.244251 4438 tablet_bootstrap.cc:492] T 9423faadbcef49c2a66144eee1991476 P bc0b21918b524c05aeada9beda0f7b75: No bootstrap required, opened a new log
I20260808 08:25:33.244339 4438 ts_tablet_manager.cc:1397] T 9423faadbcef49c2a66144eee1991476 P bc0b21918b524c05aeada9beda0f7b75: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 08:25:33.244477 4439 raft_consensus.cc:348] T 9423faadbcef49c2a66144eee1991476 P 417a70ae534e48e580b4dad4ae39dde6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } }
I20260808 08:25:33.244582 4439 raft_consensus.cc:374] T 9423faadbcef49c2a66144eee1991476 P 417a70ae534e48e580b4dad4ae39dde6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:33.244603 4439 raft_consensus.cc:729] T 9423faadbcef49c2a66144eee1991476 P 417a70ae534e48e580b4dad4ae39dde6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 417a70ae534e48e580b4dad4ae39dde6, State: Initialized, Role: FOLLOWER
I20260808 08:25:33.244751 4439 consensus_queue.cc:260] T 9423faadbcef49c2a66144eee1991476 P 417a70ae534e48e580b4dad4ae39dde6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } }
I20260808 08:25:33.244760 4438 raft_consensus.cc:348] T 9423faadbcef49c2a66144eee1991476 P bc0b21918b524c05aeada9beda0f7b75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } }
I20260808 08:25:33.244848 4438 raft_consensus.cc:374] T 9423faadbcef49c2a66144eee1991476 P bc0b21918b524c05aeada9beda0f7b75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:33.244884 4438 raft_consensus.cc:729] T 9423faadbcef49c2a66144eee1991476 P bc0b21918b524c05aeada9beda0f7b75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc0b21918b524c05aeada9beda0f7b75, State: Initialized, Role: FOLLOWER
I20260808 08:25:33.245004 4438 consensus_queue.cc:260] T 9423faadbcef49c2a66144eee1991476 P bc0b21918b524c05aeada9beda0f7b75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } }
I20260808 08:25:33.244977 4449 tablet_replica.cc:439] T 9423faadbcef49c2a66144eee1991476 P 417a70ae534e48e580b4dad4ae39dde6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } } }
I20260808 08:25:33.245005 4439 ts_tablet_manager.cc:1428] T 9423faadbcef49c2a66144eee1991476 P 417a70ae534e48e580b4dad4ae39dde6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 08:25:33.245127 4449 tablet_replica.cc:442] T 9423faadbcef49c2a66144eee1991476 P 417a70ae534e48e580b4dad4ae39dde6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 08:25:33.245249 4438 ts_tablet_manager.cc:1428] T 9423faadbcef49c2a66144eee1991476 P bc0b21918b524c05aeada9beda0f7b75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 08:25:33.245448 4448 tablet_replica.cc:439] T 9423faadbcef49c2a66144eee1991476 P bc0b21918b524c05aeada9beda0f7b75: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } } }
I20260808 08:25:33.245589 4448 tablet_replica.cc:442] T 9423faadbcef49c2a66144eee1991476 P bc0b21918b524c05aeada9beda0f7b75: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 08:25:33.245690 4437 tablet_bootstrap.cc:492] T 9423faadbcef49c2a66144eee1991476 P a89f34ab8de74b31b499851c051728cc: No bootstrap required, opened a new log
I20260808 08:25:33.245759 4437 ts_tablet_manager.cc:1397] T 9423faadbcef49c2a66144eee1991476 P a89f34ab8de74b31b499851c051728cc: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260808 08:25:33.246145 4437 raft_consensus.cc:348] T 9423faadbcef49c2a66144eee1991476 P a89f34ab8de74b31b499851c051728cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } }
I20260808 08:25:33.246248 4437 raft_consensus.cc:374] T 9423faadbcef49c2a66144eee1991476 P a89f34ab8de74b31b499851c051728cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:33.246277 4437 raft_consensus.cc:729] T 9423faadbcef49c2a66144eee1991476 P a89f34ab8de74b31b499851c051728cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a89f34ab8de74b31b499851c051728cc, State: Initialized, Role: FOLLOWER
I20260808 08:25:33.246395 4437 consensus_queue.cc:260] T 9423faadbcef49c2a66144eee1991476 P a89f34ab8de74b31b499851c051728cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } }
I20260808 08:25:33.246629 4437 ts_tablet_manager.cc:1428] T 9423faadbcef49c2a66144eee1991476 P a89f34ab8de74b31b499851c051728cc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 08:25:33.246836 4464 tablet_replica.cc:439] T 9423faadbcef49c2a66144eee1991476 P a89f34ab8de74b31b499851c051728cc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } } }
I20260808 08:25:33.246958 4464 tablet_replica.cc:442] T 9423faadbcef49c2a66144eee1991476 P a89f34ab8de74b31b499851c051728cc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 08:25:33.341064 4448 raft_consensus.cc:482] T 9423faadbcef49c2a66144eee1991476 P bc0b21918b524c05aeada9beda0f7b75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 08:25:33.341212 4448 raft_consensus.cc:504] T 9423faadbcef49c2a66144eee1991476 P bc0b21918b524c05aeada9beda0f7b75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } }
I20260808 08:25:33.342240 4448 leader_election.cc:288] T 9423faadbcef49c2a66144eee1991476 P bc0b21918b524c05aeada9beda0f7b75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 417a70ae534e48e580b4dad4ae39dde6 (127.22.235.194:34595), a89f34ab8de74b31b499851c051728cc (127.22.235.195:45553)
I20260808 08:25:33.346055 4431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:33.342528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:60621 (local address 127.22.235.194:34595)
0808 08:25:33.342657 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:33.342662 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:33.342895 (+ 233us) server_negotiation.cc:407] Connection header received
0808 08:25:33.343019 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:33.343022 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:33.343066 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:33.343129 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:25:33.343363 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:33.343856 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:33.345126 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:33.345515 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:33.345619 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:33.345678 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:33.345852 (+ 174us) server_negotiation.cc:299] Negotiation successful
0808 08:25:33.345926 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 08:25:33.346060 4436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:33.342628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:35551 (local address 127.22.235.195:45553)
0808 08:25:33.342751 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:33.342756 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:33.342940 (+ 184us) server_negotiation.cc:407] Connection header received
0808 08:25:33.343073 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:33.343075 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:33.343117 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:33.343181 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:25:33.343421 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:33.343901 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:33.345136 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:33.345523 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:33.345628 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:33.345683 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:33.345869 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 08:25:33.345940 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 08:25:33.346082 4497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:33.342454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.194:34595 (local address 127.22.235.193:60621)
0808 08:25:33.342799 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:33.342810 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:33.342916 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:33.343146 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:33.343149 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:33.343164 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:25:33.343261 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:33.343266 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:33.343955 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:33.343958 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:33.345007 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:33.345018 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:33.345619 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:33.345623 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:33.345713 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:33.345762 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 08:25:33.345826 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 08:25:33.345925 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"mutex_wait_us":80,"thread_start_us":123,"threads_started":1}
I20260808 08:25:33.346091 4496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:33.342516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.195:45553 (local address 127.22.235.193:35551)
0808 08:25:33.342850 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:33.342861 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:33.342962 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:33.343202 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:33.343205 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:33.343221 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:25:33.343319 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:33.343324 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:33.344034 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:33.344037 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:33.345009 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:33.345020 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:33.345650 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:33.345654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:33.345756 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:33.345776 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 08:25:33.345839 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 08:25:33.345931 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":42,"thread_start_us":94,"threads_started":1}
I20260808 08:25:33.346549 4075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9423faadbcef49c2a66144eee1991476" candidate_uuid: "bc0b21918b524c05aeada9beda0f7b75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "417a70ae534e48e580b4dad4ae39dde6" is_pre_election: true
I20260808 08:25:33.346699 4075 raft_consensus.cc:2454] T 9423faadbcef49c2a66144eee1991476 P 417a70ae534e48e580b4dad4ae39dde6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc0b21918b524c05aeada9beda0f7b75 in term 0.
I20260808 08:25:33.346776 4311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9423faadbcef49c2a66144eee1991476" candidate_uuid: "bc0b21918b524c05aeada9beda0f7b75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a89f34ab8de74b31b499851c051728cc" is_pre_election: true
I20260808 08:25:33.347028 4311 raft_consensus.cc:2454] T 9423faadbcef49c2a66144eee1991476 P a89f34ab8de74b31b499851c051728cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc0b21918b524c05aeada9beda0f7b75 in term 0.
I20260808 08:25:33.347096 3704 leader_election.cc:302] T 9423faadbcef49c2a66144eee1991476 P bc0b21918b524c05aeada9beda0f7b75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 417a70ae534e48e580b4dad4ae39dde6, bc0b21918b524c05aeada9beda0f7b75; no voters:
I20260808 08:25:33.347358 4448 raft_consensus.cc:2781] T 9423faadbcef49c2a66144eee1991476 P bc0b21918b524c05aeada9beda0f7b75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 08:25:33.347424 4448 raft_consensus.cc:482] T 9423faadbcef49c2a66144eee1991476 P bc0b21918b524c05aeada9beda0f7b75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 08:25:33.347462 4448 raft_consensus.cc:3037] T 9423faadbcef49c2a66144eee1991476 P bc0b21918b524c05aeada9beda0f7b75 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:33.348747 4448 raft_consensus.cc:504] T 9423faadbcef49c2a66144eee1991476 P bc0b21918b524c05aeada9beda0f7b75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } }
I20260808 08:25:33.349037 4448 leader_election.cc:288] T 9423faadbcef49c2a66144eee1991476 P bc0b21918b524c05aeada9beda0f7b75 [CANDIDATE]: Term 1 election: Requested vote from peers 417a70ae534e48e580b4dad4ae39dde6 (127.22.235.194:34595), a89f34ab8de74b31b499851c051728cc (127.22.235.195:45553)
I20260808 08:25:33.349392 4311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9423faadbcef49c2a66144eee1991476" candidate_uuid: "bc0b21918b524c05aeada9beda0f7b75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a89f34ab8de74b31b499851c051728cc"
I20260808 08:25:33.349380 4075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9423faadbcef49c2a66144eee1991476" candidate_uuid: "bc0b21918b524c05aeada9beda0f7b75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "417a70ae534e48e580b4dad4ae39dde6"
I20260808 08:25:33.349512 4075 raft_consensus.cc:3037] T 9423faadbcef49c2a66144eee1991476 P 417a70ae534e48e580b4dad4ae39dde6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:33.349519 4311 raft_consensus.cc:3037] T 9423faadbcef49c2a66144eee1991476 P a89f34ab8de74b31b499851c051728cc [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:33.350327 4075 raft_consensus.cc:2454] T 9423faadbcef49c2a66144eee1991476 P 417a70ae534e48e580b4dad4ae39dde6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc0b21918b524c05aeada9beda0f7b75 in term 1.
I20260808 08:25:33.350454 4311 raft_consensus.cc:2454] T 9423faadbcef49c2a66144eee1991476 P a89f34ab8de74b31b499851c051728cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc0b21918b524c05aeada9beda0f7b75 in term 1.
I20260808 08:25:33.350576 3704 leader_election.cc:302] T 9423faadbcef49c2a66144eee1991476 P bc0b21918b524c05aeada9beda0f7b75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 417a70ae534e48e580b4dad4ae39dde6, bc0b21918b524c05aeada9beda0f7b75; no voters:
I20260808 08:25:33.350745 4448 raft_consensus.cc:2781] T 9423faadbcef49c2a66144eee1991476 P bc0b21918b524c05aeada9beda0f7b75 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:25:33.350927 4448 raft_consensus.cc:686] T 9423faadbcef49c2a66144eee1991476 P bc0b21918b524c05aeada9beda0f7b75 [term 1 LEADER]: Becoming Leader. State: Replica: bc0b21918b524c05aeada9beda0f7b75, State: Running, Role: LEADER
I20260808 08:25:33.351145 4448 consensus_queue.cc:237] T 9423faadbcef49c2a66144eee1991476 P bc0b21918b524c05aeada9beda0f7b75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } }
I20260808 08:25:33.352186 4501 tablet_replica.cc:439] T 9423faadbcef49c2a66144eee1991476 P bc0b21918b524c05aeada9beda0f7b75: TxnStatusTablet state changed. Reason: New leader bc0b21918b524c05aeada9beda0f7b75. Latest consensus state: current_term: 1 leader_uuid: "bc0b21918b524c05aeada9beda0f7b75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } } }
I20260808 08:25:33.352298 4501 tablet_replica.cc:442] T 9423faadbcef49c2a66144eee1991476 P bc0b21918b524c05aeada9beda0f7b75: This TxnStatusTablet replica's current role is: LEADER
I20260808 08:25:33.352566 4505 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 08:25:33.352738 4505 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 08:25:33.353374 3545 catalog_manager.cc:5495] T 9423faadbcef49c2a66144eee1991476 P bc0b21918b524c05aeada9beda0f7b75 reported cstate change: term changed from 0 to 1, leader changed from <none> to bc0b21918b524c05aeada9beda0f7b75 (127.22.235.193). New cstate: current_term: 1 leader_uuid: "bc0b21918b524c05aeada9beda0f7b75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260808 08:25:33.372378 4491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:33.368672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:46367 (local address 127.0.0.1:36122)
0808 08:25:33.368805 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:33.368820 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:33.368906 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:33.369203 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:33.369205 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:33.369211 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 08:25:33.369297 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:33.369301 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:33.370233 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:33.370235 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:33.370813 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:33.370819 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:33.371345 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:33.371347 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:33.371407 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:33.371451 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 08:25:33.372159 (+ 708us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 08:25:33.372166 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 08:25:33.372202 (+ 36us) client_negotiation.cc:240] Negotiation successful
0808 08:25:33.372268 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260808 08:25:33.372404 4433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:33.368739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36122 (local address 127.22.235.193:46367)
0808 08:25:33.368857 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:33.368861 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:33.368877 (+ 16us) server_negotiation.cc:407] Connection header received
0808 08:25:33.369037 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:33.369041 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:33.369101 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:33.369203 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:25:33.369379 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:33.370145 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:33.370905 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:33.371256 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:33.371356 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:33.371568 (+ 212us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:25:33.372082 (+ 514us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 08:25:33.372188 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:33.372219 (+ 31us) server_negotiation.cc:299] Negotiation successful
0808 08:25:33.372273 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 08:25:33.375145 4075 raft_consensus.cc:1261] T 9423faadbcef49c2a66144eee1991476 P 417a70ae534e48e580b4dad4ae39dde6 [term 1 FOLLOWER]: Refusing update from remote peer bc0b21918b524c05aeada9beda0f7b75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:25:33.375273 4311 raft_consensus.cc:1261] T 9423faadbcef49c2a66144eee1991476 P a89f34ab8de74b31b499851c051728cc [term 1 FOLLOWER]: Refusing update from remote peer bc0b21918b524c05aeada9beda0f7b75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:25:33.375625 4448 consensus_queue.cc:1035] T 9423faadbcef49c2a66144eee1991476 P bc0b21918b524c05aeada9beda0f7b75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:25:33.375780 4501 consensus_queue.cc:1035] T 9423faadbcef49c2a66144eee1991476 P bc0b21918b524c05aeada9beda0f7b75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:25:33.377167 4449 tablet_replica.cc:439] T 9423faadbcef49c2a66144eee1991476 P 417a70ae534e48e580b4dad4ae39dde6: TxnStatusTablet state changed. Reason: New leader bc0b21918b524c05aeada9beda0f7b75. Latest consensus state: current_term: 1 leader_uuid: "bc0b21918b524c05aeada9beda0f7b75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } } }
I20260808 08:25:33.377281 4449 tablet_replica.cc:442] T 9423faadbcef49c2a66144eee1991476 P 417a70ae534e48e580b4dad4ae39dde6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 08:25:33.377513 4464 tablet_replica.cc:439] T 9423faadbcef49c2a66144eee1991476 P a89f34ab8de74b31b499851c051728cc: TxnStatusTablet state changed. Reason: New leader bc0b21918b524c05aeada9beda0f7b75. Latest consensus state: current_term: 1 leader_uuid: "bc0b21918b524c05aeada9beda0f7b75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } } }
I20260808 08:25:33.377620 4464 tablet_replica.cc:442] T 9423faadbcef49c2a66144eee1991476 P a89f34ab8de74b31b499851c051728cc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 08:25:33.379024 4448 tablet_replica.cc:439] T 9423faadbcef49c2a66144eee1991476 P bc0b21918b524c05aeada9beda0f7b75: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bc0b21918b524c05aeada9beda0f7b75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } } }
I20260808 08:25:33.379130 4448 tablet_replica.cc:442] T 9423faadbcef49c2a66144eee1991476 P bc0b21918b524c05aeada9beda0f7b75: This TxnStatusTablet replica's current role is: LEADER
I20260808 08:25:33.379731 4501 tablet_replica.cc:439] T 9423faadbcef49c2a66144eee1991476 P bc0b21918b524c05aeada9beda0f7b75: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bc0b21918b524c05aeada9beda0f7b75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } } }
I20260808 08:25:33.379827 4501 tablet_replica.cc:442] T 9423faadbcef49c2a66144eee1991476 P bc0b21918b524c05aeada9beda0f7b75: This TxnStatusTablet replica's current role is: LEADER
I20260808 08:25:33.380900 4449 tablet_replica.cc:439] T 9423faadbcef49c2a66144eee1991476 P 417a70ae534e48e580b4dad4ae39dde6: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bc0b21918b524c05aeada9beda0f7b75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } } }
I20260808 08:25:33.381016 4449 tablet_replica.cc:442] T 9423faadbcef49c2a66144eee1991476 P 417a70ae534e48e580b4dad4ae39dde6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 08:25:33.381937 4464 tablet_replica.cc:439] T 9423faadbcef49c2a66144eee1991476 P a89f34ab8de74b31b499851c051728cc: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bc0b21918b524c05aeada9beda0f7b75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } } }
I20260808 08:25:33.382046 4464 tablet_replica.cc:442] T 9423faadbcef49c2a66144eee1991476 P a89f34ab8de74b31b499851c051728cc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 08:25:33.394018 4433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:33.390792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:42331 (local address 127.22.235.193:46367)
0808 08:25:33.390897 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:33.390900 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:33.390949 (+ 49us) server_negotiation.cc:407] Connection header received
0808 08:25:33.391098 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:33.391100 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:33.391142 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:33.391200 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:25:33.391430 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:33.391906 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:33.393101 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:33.393309 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:33.393383 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:33.393664 (+ 281us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:25:33.393729 (+ 65us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 08:25:33.393778 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:33.393864 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 08:25:33.393906 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 08:25:33.394093 4519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:33.390647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:46367 (local address 127.22.235.195:42331)
0808 08:25:33.390879 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:33.390894 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:33.391011 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:33.391228 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:33.391231 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:33.391240 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 08:25:33.391345 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:33.391352 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:33.392006 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:33.392009 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:33.392997 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:33.393009 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:33.393403 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:33.393407 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:33.393493 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:33.393550 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 08:25:33.393795 (+ 245us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 08:25:33.393801 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 08:25:33.393852 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 08:25:33.393936 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":53,"threads_started":1}
I20260808 08:25:33.400251 4075 raft_consensus.cc:1261] T 2338a2e248b34eb495819568927ed019 P 417a70ae534e48e580b4dad4ae39dde6 [term 1 FOLLOWER]: Refusing update from remote peer a89f34ab8de74b31b499851c051728cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:25:33.400274 3818 raft_consensus.cc:1261] T 2338a2e248b34eb495819568927ed019 P bc0b21918b524c05aeada9beda0f7b75 [term 1 FOLLOWER]: Refusing update from remote peer a89f34ab8de74b31b499851c051728cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:25:33.400693 4464 consensus_queue.cc:1035] T 2338a2e248b34eb495819568927ed019 P a89f34ab8de74b31b499851c051728cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:25:33.400838 4447 consensus_queue.cc:1035] T 2338a2e248b34eb495819568927ed019 P a89f34ab8de74b31b499851c051728cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:25:33.422176 4436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:33.418534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:53515 (local address 127.22.235.195:45553)
0808 08:25:33.418644 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:33.418648 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:33.418918 (+ 270us) server_negotiation.cc:407] Connection header received
0808 08:25:33.419130 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:33.419134 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:33.419184 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:33.419248 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:25:33.419537 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:33.420169 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:33.421071 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:33.421284 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:33.421369 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:33.421709 (+ 340us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:25:33.421792 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 08:25:33.421863 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:33.421985 (+ 122us) server_negotiation.cc:299] Negotiation successful
0808 08:25:33.422050 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 08:25:33.422227 4528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:33.418504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.195:45553 (local address 127.22.235.193:53515)
0808 08:25:33.418842 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:33.418856 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:33.419009 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:33.419287 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:33.419290 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:33.419298 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 08:25:33.419427 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:33.419434 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:33.420270 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:33.420274 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:33.420966 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:33.420976 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:33.421376 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:33.421379 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:33.421465 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:33.421546 (+ 81us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 08:25:33.421884 (+ 338us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 08:25:33.421892 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 08:25:33.421964 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 08:25:33.422042 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":92,"threads_started":1}
I20260808 08:25:33.444684 3544 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:25:33.444867 3544 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:25:33.446727 3544 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a626e0cf38834d9992e58716fe7ae506: Sending DeleteTablet for 3 replicas of tablet 2338a2e248b34eb495819568927ed019
I20260808 08:25:33.447279 4272 tablet_service.cc:1514] Processing DeleteTablet for tablet 2338a2e248b34eb495819568927ed019 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:25:33 UTC) from {username='oryx', principal='oryx/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:36776
I20260808 08:25:33.447310 4026 tablet_service.cc:1514] Processing DeleteTablet for tablet 2338a2e248b34eb495819568927ed019 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:25:33 UTC) from {username='oryx', principal='oryx/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:55482
I20260808 08:25:33.447448 4473 tablet_replica.cc:330] T 2338a2e248b34eb495819568927ed019 P a89f34ab8de74b31b499851c051728cc: stopping tablet replica
I20260808 08:25:33.447579 4473 raft_consensus.cc:2229] T 2338a2e248b34eb495819568927ed019 P a89f34ab8de74b31b499851c051728cc [term 1 LEADER]: Raft consensus shutting down.
I20260808 08:25:33.447739 4473 raft_consensus.cc:2258] T 2338a2e248b34eb495819568927ed019 P a89f34ab8de74b31b499851c051728cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:25:33.447726 3784 tablet_service.cc:1514] Processing DeleteTablet for tablet 2338a2e248b34eb495819568927ed019 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:25:33 UTC) from {username='oryx', principal='oryx/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:36110
I20260808 08:25:33.447904 4471 tablet_replica.cc:330] T 2338a2e248b34eb495819568927ed019 P bc0b21918b524c05aeada9beda0f7b75: stopping tablet replica
I20260808 08:25:33.448004 4471 raft_consensus.cc:2229] T 2338a2e248b34eb495819568927ed019 P bc0b21918b524c05aeada9beda0f7b75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 08:25:33.448093 4471 raft_consensus.cc:2258] T 2338a2e248b34eb495819568927ed019 P bc0b21918b524c05aeada9beda0f7b75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:25:33.448283 4530 tablet_replica.cc:330] T 2338a2e248b34eb495819568927ed019 P 417a70ae534e48e580b4dad4ae39dde6: stopping tablet replica
I20260808 08:25:33.448387 4530 raft_consensus.cc:2229] T 2338a2e248b34eb495819568927ed019 P 417a70ae534e48e580b4dad4ae39dde6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 08:25:33.448400 4473 ts_tablet_manager.cc:1905] T 2338a2e248b34eb495819568927ed019 P a89f34ab8de74b31b499851c051728cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:25:33.448534 4530 raft_consensus.cc:2258] T 2338a2e248b34eb495819568927ed019 P 417a70ae534e48e580b4dad4ae39dde6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:25:33.448835 4471 ts_tablet_manager.cc:1905] T 2338a2e248b34eb495819568927ed019 P bc0b21918b524c05aeada9beda0f7b75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:25:33.451331 4471 ts_tablet_manager.cc:1918] T 2338a2e248b34eb495819568927ed019 P bc0b21918b524c05aeada9beda0f7b75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 08:25:33.451336 4473 ts_tablet_manager.cc:1918] T 2338a2e248b34eb495819568927ed019 P a89f34ab8de74b31b499851c051728cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 08:25:33.451419 4473 log.cc:1196] T 2338a2e248b34eb495819568927ed019 P a89f34ab8de74b31b499851c051728cc: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-2/wal/wals/2338a2e248b34eb495819568927ed019
I20260808 08:25:33.451421 4471 log.cc:1196] T 2338a2e248b34eb495819568927ed019 P bc0b21918b524c05aeada9beda0f7b75: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-0/wal/wals/2338a2e248b34eb495819568927ed019
I20260808 08:25:33.451683 4471 ts_tablet_manager.cc:1939] T 2338a2e248b34eb495819568927ed019 P bc0b21918b524c05aeada9beda0f7b75: Deleting consensus metadata
I20260808 08:25:33.451700 4473 ts_tablet_manager.cc:1939] T 2338a2e248b34eb495819568927ed019 P a89f34ab8de74b31b499851c051728cc: Deleting consensus metadata
I20260808 08:25:33.452385 3522 catalog_manager.cc:4841] TS bc0b21918b524c05aeada9beda0f7b75 (127.22.235.193:46367): tablet 2338a2e248b34eb495819568927ed019 (table test-table [id=44bdf8fa616445d8a3eda472490a007a]) successfully deleted
I20260808 08:25:33.452522 3520 catalog_manager.cc:4841] TS a89f34ab8de74b31b499851c051728cc (127.22.235.195:45553): tablet 2338a2e248b34eb495819568927ed019 (table test-table [id=44bdf8fa616445d8a3eda472490a007a]) successfully deleted
I20260808 08:25:33.453097 4530 ts_tablet_manager.cc:1905] T 2338a2e248b34eb495819568927ed019 P 417a70ae534e48e580b4dad4ae39dde6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:25:33.455992 4530 ts_tablet_manager.cc:1918] T 2338a2e248b34eb495819568927ed019 P 417a70ae534e48e580b4dad4ae39dde6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 08:25:33.456074 4530 log.cc:1196] T 2338a2e248b34eb495819568927ed019 P 417a70ae534e48e580b4dad4ae39dde6: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-1/wal/wals/2338a2e248b34eb495819568927ed019
I20260808 08:25:33.456380 4530 ts_tablet_manager.cc:1939] T 2338a2e248b34eb495819568927ed019 P 417a70ae534e48e580b4dad4ae39dde6: Deleting consensus metadata
I20260808 08:25:33.457088 3524 catalog_manager.cc:4841] TS 417a70ae534e48e580b4dad4ae39dde6 (127.22.235.194:34595): tablet 2338a2e248b34eb495819568927ed019 (table test-table [id=44bdf8fa616445d8a3eda472490a007a]) successfully deleted
Aug 08 08:25:33 dist-test-slave-f7th krb5kdc[3469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 08:25:33 dist-test-slave-f7th krb5kdc[3469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:33 dist-test-slave-f7th krb5kdc[3469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
W20260808 08:25:33.476401 3627 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:33.470256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53874 (local address 127.22.235.254:36595)
0808 08:25:33.470380 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:33.470384 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:33.470580 (+ 196us) server_negotiation.cc:407] Connection header received
0808 08:25:33.470730 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:33.470733 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:33.470780 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:33.470845 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:33.471552 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:33.472088 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:33.473020 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:33.473244 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:33.473332 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:33.476277 (+ 2945us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53874: BlockingRecv error: recv got EOF from 127.0.0.1:53874 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 08 08:25:33 dist-test-slave-f7th krb5kdc[3469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM
I20260808 08:25:33.486608 3627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:33.478046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53876 (local address 127.22.235.254:36595)
0808 08:25:33.478190 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:33.478194 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:33.478474 (+ 280us) server_negotiation.cc:407] Connection header received
0808 08:25:33.478640 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:33.478643 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:33.478702 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:33.478764 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:33.479364 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:33.479934 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:33.480730 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:33.480934 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:33.481022 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:33.483477 (+ 2455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:33.483482 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:33.483485 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:33.483523 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:33.485009 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:33.485452 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:33.485454 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:33.485455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:33.485551 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:33.485855 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:33.485857 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:33.485859 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:33.486028 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:33.486125 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:33.486366 (+ 241us) server_negotiation.cc:299] Negotiation successful
0808 08:25:33.486455 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 08:25:33.489161 3544 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53876:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 08:25:33.489593 3544 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 08:25:33.494275 4026 tablet_service.cc:1467] Processing CreateTablet for tablet 71c38d1419c04215a0051aebb9a525c9 (DEFAULT_TABLE table=test-table [id=144d8de6206148c492ececb6519f3fff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:25:33.494287 4272 tablet_service.cc:1467] Processing CreateTablet for tablet 71c38d1419c04215a0051aebb9a525c9 (DEFAULT_TABLE table=test-table [id=144d8de6206148c492ececb6519f3fff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:25:33.494277 3784 tablet_service.cc:1467] Processing CreateTablet for tablet 71c38d1419c04215a0051aebb9a525c9 (DEFAULT_TABLE table=test-table [id=144d8de6206148c492ececb6519f3fff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:25:33.494589 4272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71c38d1419c04215a0051aebb9a525c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:33.494590 4026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71c38d1419c04215a0051aebb9a525c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:33.494596 3784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71c38d1419c04215a0051aebb9a525c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:33.496976 4437 tablet_bootstrap.cc:492] T 71c38d1419c04215a0051aebb9a525c9 P a89f34ab8de74b31b499851c051728cc: Bootstrap starting.
I20260808 08:25:33.496968 4439 tablet_bootstrap.cc:492] T 71c38d1419c04215a0051aebb9a525c9 P 417a70ae534e48e580b4dad4ae39dde6: Bootstrap starting.
I20260808 08:25:33.497819 4438 tablet_bootstrap.cc:492] T 71c38d1419c04215a0051aebb9a525c9 P bc0b21918b524c05aeada9beda0f7b75: Bootstrap starting.
I20260808 08:25:33.497862 4439 tablet_bootstrap.cc:654] T 71c38d1419c04215a0051aebb9a525c9 P 417a70ae534e48e580b4dad4ae39dde6: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:33.497861 4437 tablet_bootstrap.cc:654] T 71c38d1419c04215a0051aebb9a525c9 P a89f34ab8de74b31b499851c051728cc: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:33.498731 4438 tablet_bootstrap.cc:654] T 71c38d1419c04215a0051aebb9a525c9 P bc0b21918b524c05aeada9beda0f7b75: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:33.499357 4439 tablet_bootstrap.cc:492] T 71c38d1419c04215a0051aebb9a525c9 P 417a70ae534e48e580b4dad4ae39dde6: No bootstrap required, opened a new log
I20260808 08:25:33.499444 4439 ts_tablet_manager.cc:1397] T 71c38d1419c04215a0051aebb9a525c9 P 417a70ae534e48e580b4dad4ae39dde6: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 08:25:33.499930 4439 raft_consensus.cc:348] T 71c38d1419c04215a0051aebb9a525c9 P 417a70ae534e48e580b4dad4ae39dde6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } }
I20260808 08:25:33.500051 4439 raft_consensus.cc:374] T 71c38d1419c04215a0051aebb9a525c9 P 417a70ae534e48e580b4dad4ae39dde6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:33.500082 4439 raft_consensus.cc:729] T 71c38d1419c04215a0051aebb9a525c9 P 417a70ae534e48e580b4dad4ae39dde6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 417a70ae534e48e580b4dad4ae39dde6, State: Initialized, Role: FOLLOWER
I20260808 08:25:33.500233 4439 consensus_queue.cc:260] T 71c38d1419c04215a0051aebb9a525c9 P 417a70ae534e48e580b4dad4ae39dde6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } }
I20260808 08:25:33.500430 4437 tablet_bootstrap.cc:492] T 71c38d1419c04215a0051aebb9a525c9 P a89f34ab8de74b31b499851c051728cc: No bootstrap required, opened a new log
I20260808 08:25:33.500488 4438 tablet_bootstrap.cc:492] T 71c38d1419c04215a0051aebb9a525c9 P bc0b21918b524c05aeada9beda0f7b75: No bootstrap required, opened a new log
I20260808 08:25:33.500504 4437 ts_tablet_manager.cc:1397] T 71c38d1419c04215a0051aebb9a525c9 P a89f34ab8de74b31b499851c051728cc: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260808 08:25:33.500531 4439 ts_tablet_manager.cc:1428] T 71c38d1419c04215a0051aebb9a525c9 P 417a70ae534e48e580b4dad4ae39dde6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 08:25:33.500569 4438 ts_tablet_manager.cc:1397] T 71c38d1419c04215a0051aebb9a525c9 P bc0b21918b524c05aeada9beda0f7b75: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 08:25:33.500963 4437 raft_consensus.cc:348] T 71c38d1419c04215a0051aebb9a525c9 P a89f34ab8de74b31b499851c051728cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } }
I20260808 08:25:33.501077 4437 raft_consensus.cc:374] T 71c38d1419c04215a0051aebb9a525c9 P a89f34ab8de74b31b499851c051728cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:33.501108 4437 raft_consensus.cc:729] T 71c38d1419c04215a0051aebb9a525c9 P a89f34ab8de74b31b499851c051728cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a89f34ab8de74b31b499851c051728cc, State: Initialized, Role: FOLLOWER
I20260808 08:25:33.501092 4438 raft_consensus.cc:348] T 71c38d1419c04215a0051aebb9a525c9 P bc0b21918b524c05aeada9beda0f7b75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } }
I20260808 08:25:33.501205 4438 raft_consensus.cc:374] T 71c38d1419c04215a0051aebb9a525c9 P bc0b21918b524c05aeada9beda0f7b75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:33.501237 4438 raft_consensus.cc:729] T 71c38d1419c04215a0051aebb9a525c9 P bc0b21918b524c05aeada9beda0f7b75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc0b21918b524c05aeada9beda0f7b75, State: Initialized, Role: FOLLOWER
I20260808 08:25:33.501220 4437 consensus_queue.cc:260] T 71c38d1419c04215a0051aebb9a525c9 P a89f34ab8de74b31b499851c051728cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } }
I20260808 08:25:33.501360 4438 consensus_queue.cc:260] T 71c38d1419c04215a0051aebb9a525c9 P bc0b21918b524c05aeada9beda0f7b75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } }
I20260808 08:25:33.501457 4437 ts_tablet_manager.cc:1428] T 71c38d1419c04215a0051aebb9a525c9 P a89f34ab8de74b31b499851c051728cc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 08:25:33.501626 4438 ts_tablet_manager.cc:1428] T 71c38d1419c04215a0051aebb9a525c9 P bc0b21918b524c05aeada9beda0f7b75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 08:25:33.561158 4522 raft_consensus.cc:482] T 71c38d1419c04215a0051aebb9a525c9 P bc0b21918b524c05aeada9beda0f7b75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 08:25:33.561305 4522 raft_consensus.cc:504] T 71c38d1419c04215a0051aebb9a525c9 P bc0b21918b524c05aeada9beda0f7b75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } }
I20260808 08:25:33.561790 4522 leader_election.cc:288] T 71c38d1419c04215a0051aebb9a525c9 P bc0b21918b524c05aeada9beda0f7b75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 417a70ae534e48e580b4dad4ae39dde6 (127.22.235.194:34595), a89f34ab8de74b31b499851c051728cc (127.22.235.195:45553)
I20260808 08:25:33.562181 4075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71c38d1419c04215a0051aebb9a525c9" candidate_uuid: "bc0b21918b524c05aeada9beda0f7b75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "417a70ae534e48e580b4dad4ae39dde6" is_pre_election: true
I20260808 08:25:33.562224 4311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71c38d1419c04215a0051aebb9a525c9" candidate_uuid: "bc0b21918b524c05aeada9beda0f7b75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a89f34ab8de74b31b499851c051728cc" is_pre_election: true
I20260808 08:25:33.562315 4075 raft_consensus.cc:2454] T 71c38d1419c04215a0051aebb9a525c9 P 417a70ae534e48e580b4dad4ae39dde6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc0b21918b524c05aeada9beda0f7b75 in term 0.
I20260808 08:25:33.562358 4311 raft_consensus.cc:2454] T 71c38d1419c04215a0051aebb9a525c9 P a89f34ab8de74b31b499851c051728cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc0b21918b524c05aeada9beda0f7b75 in term 0.
I20260808 08:25:33.562604 3704 leader_election.cc:302] T 71c38d1419c04215a0051aebb9a525c9 P bc0b21918b524c05aeada9beda0f7b75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 417a70ae534e48e580b4dad4ae39dde6, bc0b21918b524c05aeada9beda0f7b75; no voters:
I20260808 08:25:33.562794 4522 raft_consensus.cc:2781] T 71c38d1419c04215a0051aebb9a525c9 P bc0b21918b524c05aeada9beda0f7b75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 08:25:33.562857 4522 raft_consensus.cc:482] T 71c38d1419c04215a0051aebb9a525c9 P bc0b21918b524c05aeada9beda0f7b75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 08:25:33.562885 4522 raft_consensus.cc:3037] T 71c38d1419c04215a0051aebb9a525c9 P bc0b21918b524c05aeada9beda0f7b75 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:33.563864 4522 raft_consensus.cc:504] T 71c38d1419c04215a0051aebb9a525c9 P bc0b21918b524c05aeada9beda0f7b75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } }
I20260808 08:25:33.564178 4522 leader_election.cc:288] T 71c38d1419c04215a0051aebb9a525c9 P bc0b21918b524c05aeada9beda0f7b75 [CANDIDATE]: Term 1 election: Requested vote from peers 417a70ae534e48e580b4dad4ae39dde6 (127.22.235.194:34595), a89f34ab8de74b31b499851c051728cc (127.22.235.195:45553)
I20260808 08:25:33.564523 4075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71c38d1419c04215a0051aebb9a525c9" candidate_uuid: "bc0b21918b524c05aeada9beda0f7b75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "417a70ae534e48e580b4dad4ae39dde6"
I20260808 08:25:33.564523 4311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71c38d1419c04215a0051aebb9a525c9" candidate_uuid: "bc0b21918b524c05aeada9beda0f7b75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a89f34ab8de74b31b499851c051728cc"
I20260808 08:25:33.564667 4311 raft_consensus.cc:3037] T 71c38d1419c04215a0051aebb9a525c9 P a89f34ab8de74b31b499851c051728cc [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:33.564664 4075 raft_consensus.cc:3037] T 71c38d1419c04215a0051aebb9a525c9 P 417a70ae534e48e580b4dad4ae39dde6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:33.565722 4311 raft_consensus.cc:2454] T 71c38d1419c04215a0051aebb9a525c9 P a89f34ab8de74b31b499851c051728cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc0b21918b524c05aeada9beda0f7b75 in term 1.
I20260808 08:25:33.565722 4075 raft_consensus.cc:2454] T 71c38d1419c04215a0051aebb9a525c9 P 417a70ae534e48e580b4dad4ae39dde6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc0b21918b524c05aeada9beda0f7b75 in term 1.
I20260808 08:25:33.566012 3704 leader_election.cc:302] T 71c38d1419c04215a0051aebb9a525c9 P bc0b21918b524c05aeada9beda0f7b75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 417a70ae534e48e580b4dad4ae39dde6, a89f34ab8de74b31b499851c051728cc, bc0b21918b524c05aeada9beda0f7b75; no voters:
I20260808 08:25:33.566180 4522 raft_consensus.cc:2781] T 71c38d1419c04215a0051aebb9a525c9 P bc0b21918b524c05aeada9beda0f7b75 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:25:33.566284 4522 raft_consensus.cc:686] T 71c38d1419c04215a0051aebb9a525c9 P bc0b21918b524c05aeada9beda0f7b75 [term 1 LEADER]: Becoming Leader. State: Replica: bc0b21918b524c05aeada9beda0f7b75, State: Running, Role: LEADER
I20260808 08:25:33.566423 4522 consensus_queue.cc:237] T 71c38d1419c04215a0051aebb9a525c9 P bc0b21918b524c05aeada9beda0f7b75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } }
I20260808 08:25:33.568077 3542 catalog_manager.cc:5495] T 71c38d1419c04215a0051aebb9a525c9 P bc0b21918b524c05aeada9beda0f7b75 reported cstate change: term changed from 0 to 1, leader changed from <none> to bc0b21918b524c05aeada9beda0f7b75 (127.22.235.193). New cstate: current_term: 1 leader_uuid: "bc0b21918b524c05aeada9beda0f7b75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260808 08:25:33.583163 4433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:33.580595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36124 (local address 127.22.235.193:46367)
0808 08:25:33.580729 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:33.580733 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:33.580745 (+ 12us) server_negotiation.cc:407] Connection header received
0808 08:25:33.580804 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:33.580807 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:33.580849 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:33.580907 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:25:33.581116 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:33.581598 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:33.582372 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:33.582579 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:33.582647 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:33.582833 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:25:33.582897 (+ 64us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 08:25:33.582942 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:33.583016 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 08:25:33.583060 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 08:25:33.585366 4311 raft_consensus.cc:1261] T 71c38d1419c04215a0051aebb9a525c9 P a89f34ab8de74b31b499851c051728cc [term 1 FOLLOWER]: Refusing update from remote peer bc0b21918b524c05aeada9beda0f7b75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:25:33.585383 4075 raft_consensus.cc:1261] T 71c38d1419c04215a0051aebb9a525c9 P 417a70ae534e48e580b4dad4ae39dde6 [term 1 FOLLOWER]: Refusing update from remote peer bc0b21918b524c05aeada9beda0f7b75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:25:33.585762 4522 consensus_queue.cc:1035] T 71c38d1419c04215a0051aebb9a525c9 P bc0b21918b524c05aeada9beda0f7b75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:25:33.585917 4501 consensus_queue.cc:1035] T 71c38d1419c04215a0051aebb9a525c9 P bc0b21918b524c05aeada9beda0f7b75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:25:33.594813 3544 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:25:33.594977 3544 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:25:33.596904 3544 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a626e0cf38834d9992e58716fe7ae506: Sending DeleteTablet for 3 replicas of tablet 71c38d1419c04215a0051aebb9a525c9
I20260808 08:25:33.597514 3784 tablet_service.cc:1514] Processing DeleteTablet for tablet 71c38d1419c04215a0051aebb9a525c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:25:33 UTC) from {username='oryx', principal='oryx/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:36110
I20260808 08:25:33.597517 4272 tablet_service.cc:1514] Processing DeleteTablet for tablet 71c38d1419c04215a0051aebb9a525c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:25:33 UTC) from {username='oryx', principal='oryx/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:36776
I20260808 08:25:33.597515 4026 tablet_service.cc:1514] Processing DeleteTablet for tablet 71c38d1419c04215a0051aebb9a525c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:25:33 UTC) from {username='oryx', principal='oryx/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:55482
I20260808 08:25:33.597682 4473 tablet_replica.cc:330] T 71c38d1419c04215a0051aebb9a525c9 P a89f34ab8de74b31b499851c051728cc: stopping tablet replica
I20260808 08:25:33.597705 4530 tablet_replica.cc:330] T 71c38d1419c04215a0051aebb9a525c9 P 417a70ae534e48e580b4dad4ae39dde6: stopping tablet replica
I20260808 08:25:33.597791 4473 raft_consensus.cc:2229] T 71c38d1419c04215a0051aebb9a525c9 P a89f34ab8de74b31b499851c051728cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 08:25:33.597817 4530 raft_consensus.cc:2229] T 71c38d1419c04215a0051aebb9a525c9 P 417a70ae534e48e580b4dad4ae39dde6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 08:25:33.597913 4473 raft_consensus.cc:2258] T 71c38d1419c04215a0051aebb9a525c9 P a89f34ab8de74b31b499851c051728cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:25:33.597932 4530 raft_consensus.cc:2258] T 71c38d1419c04215a0051aebb9a525c9 P 417a70ae534e48e580b4dad4ae39dde6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:25:33.598099 4471 tablet_replica.cc:330] T 71c38d1419c04215a0051aebb9a525c9 P bc0b21918b524c05aeada9beda0f7b75: stopping tablet replica
I20260808 08:25:33.598207 4471 raft_consensus.cc:2229] T 71c38d1419c04215a0051aebb9a525c9 P bc0b21918b524c05aeada9beda0f7b75 [term 1 LEADER]: Raft consensus shutting down.
I20260808 08:25:33.598485 4471 raft_consensus.cc:2258] T 71c38d1419c04215a0051aebb9a525c9 P bc0b21918b524c05aeada9beda0f7b75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:25:33.598826 3544 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53876:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 08:25:33.599179 4530 ts_tablet_manager.cc:1905] T 71c38d1419c04215a0051aebb9a525c9 P 417a70ae534e48e580b4dad4ae39dde6: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 08:25:33.599220 3544 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 08:25:33.599300 4471 ts_tablet_manager.cc:1905] T 71c38d1419c04215a0051aebb9a525c9 P bc0b21918b524c05aeada9beda0f7b75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:25:33.601696 4530 ts_tablet_manager.cc:1918] T 71c38d1419c04215a0051aebb9a525c9 P 417a70ae534e48e580b4dad4ae39dde6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 08:25:33.601712 4471 ts_tablet_manager.cc:1918] T 71c38d1419c04215a0051aebb9a525c9 P bc0b21918b524c05aeada9beda0f7b75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 08:25:33.601780 4471 log.cc:1196] T 71c38d1419c04215a0051aebb9a525c9 P bc0b21918b524c05aeada9beda0f7b75: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-0/wal/wals/71c38d1419c04215a0051aebb9a525c9
I20260808 08:25:33.601783 4530 log.cc:1196] T 71c38d1419c04215a0051aebb9a525c9 P 417a70ae534e48e580b4dad4ae39dde6: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-1/wal/wals/71c38d1419c04215a0051aebb9a525c9
I20260808 08:25:33.602036 4471 ts_tablet_manager.cc:1939] T 71c38d1419c04215a0051aebb9a525c9 P bc0b21918b524c05aeada9beda0f7b75: Deleting consensus metadata
I20260808 08:25:33.602043 4530 ts_tablet_manager.cc:1939] T 71c38d1419c04215a0051aebb9a525c9 P 417a70ae534e48e580b4dad4ae39dde6: Deleting consensus metadata
I20260808 08:25:33.602701 3524 catalog_manager.cc:4841] TS 417a70ae534e48e580b4dad4ae39dde6 (127.22.235.194:34595): tablet 71c38d1419c04215a0051aebb9a525c9 (table test-table [id=144d8de6206148c492ececb6519f3fff]) successfully deleted
I20260808 08:25:33.602702 3522 catalog_manager.cc:4841] TS bc0b21918b524c05aeada9beda0f7b75 (127.22.235.193:46367): tablet 71c38d1419c04215a0051aebb9a525c9 (table test-table [id=144d8de6206148c492ececb6519f3fff]) successfully deleted
I20260808 08:25:33.603900 4026 tablet_service.cc:1467] Processing CreateTablet for tablet 96e4a02ff77f47c9b543e15d0860c959 (DEFAULT_TABLE table=test-table [id=bb4577859e6b425196d70954115a5125]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:25:33.603911 4272 tablet_service.cc:1467] Processing CreateTablet for tablet 96e4a02ff77f47c9b543e15d0860c959 (DEFAULT_TABLE table=test-table [id=bb4577859e6b425196d70954115a5125]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:25:33.604138 4026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96e4a02ff77f47c9b543e15d0860c959. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:33.604198 4272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96e4a02ff77f47c9b543e15d0860c959. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:33.605140 4473 ts_tablet_manager.cc:1905] T 71c38d1419c04215a0051aebb9a525c9 P a89f34ab8de74b31b499851c051728cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:25:33.606251 4439 tablet_bootstrap.cc:492] T 96e4a02ff77f47c9b543e15d0860c959 P 417a70ae534e48e580b4dad4ae39dde6: Bootstrap starting.
I20260808 08:25:33.606935 3784 tablet_service.cc:1467] Processing CreateTablet for tablet 96e4a02ff77f47c9b543e15d0860c959 (DEFAULT_TABLE table=test-table [id=bb4577859e6b425196d70954115a5125]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:25:33.607163 4439 tablet_bootstrap.cc:654] T 96e4a02ff77f47c9b543e15d0860c959 P 417a70ae534e48e580b4dad4ae39dde6: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:33.607177 3784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96e4a02ff77f47c9b543e15d0860c959. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:33.607296 4473 ts_tablet_manager.cc:1918] T 71c38d1419c04215a0051aebb9a525c9 P a89f34ab8de74b31b499851c051728cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 08:25:33.607362 4473 log.cc:1196] T 71c38d1419c04215a0051aebb9a525c9 P a89f34ab8de74b31b499851c051728cc: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-2/wal/wals/71c38d1419c04215a0051aebb9a525c9
I20260808 08:25:33.607586 4473 ts_tablet_manager.cc:1939] T 71c38d1419c04215a0051aebb9a525c9 P a89f34ab8de74b31b499851c051728cc: Deleting consensus metadata
I20260808 08:25:33.607766 4437 tablet_bootstrap.cc:492] T 96e4a02ff77f47c9b543e15d0860c959 P a89f34ab8de74b31b499851c051728cc: Bootstrap starting.
I20260808 08:25:33.608418 4437 tablet_bootstrap.cc:654] T 96e4a02ff77f47c9b543e15d0860c959 P a89f34ab8de74b31b499851c051728cc: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:33.609157 3520 catalog_manager.cc:4841] TS a89f34ab8de74b31b499851c051728cc (127.22.235.195:45553): tablet 71c38d1419c04215a0051aebb9a525c9 (table test-table [id=144d8de6206148c492ececb6519f3fff]) successfully deleted
I20260808 08:25:33.609284 4438 tablet_bootstrap.cc:492] T 96e4a02ff77f47c9b543e15d0860c959 P bc0b21918b524c05aeada9beda0f7b75: Bootstrap starting.
I20260808 08:25:33.609396 4437 tablet_bootstrap.cc:492] T 96e4a02ff77f47c9b543e15d0860c959 P a89f34ab8de74b31b499851c051728cc: No bootstrap required, opened a new log
I20260808 08:25:33.609467 4437 ts_tablet_manager.cc:1397] T 96e4a02ff77f47c9b543e15d0860c959 P a89f34ab8de74b31b499851c051728cc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 08:25:33.609827 4437 raft_consensus.cc:348] T 96e4a02ff77f47c9b543e15d0860c959 P a89f34ab8de74b31b499851c051728cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } }
I20260808 08:25:33.609943 4437 raft_consensus.cc:374] T 96e4a02ff77f47c9b543e15d0860c959 P a89f34ab8de74b31b499851c051728cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:33.609984 4437 raft_consensus.cc:729] T 96e4a02ff77f47c9b543e15d0860c959 P a89f34ab8de74b31b499851c051728cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a89f34ab8de74b31b499851c051728cc, State: Initialized, Role: FOLLOWER
I20260808 08:25:33.610042 4438 tablet_bootstrap.cc:654] T 96e4a02ff77f47c9b543e15d0860c959 P bc0b21918b524c05aeada9beda0f7b75: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:33.610121 4437 consensus_queue.cc:260] T 96e4a02ff77f47c9b543e15d0860c959 P a89f34ab8de74b31b499851c051728cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } }
I20260808 08:25:33.610381 4437 ts_tablet_manager.cc:1428] T 96e4a02ff77f47c9b543e15d0860c959 P a89f34ab8de74b31b499851c051728cc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 08:25:33.610512 4439 tablet_bootstrap.cc:492] T 96e4a02ff77f47c9b543e15d0860c959 P 417a70ae534e48e580b4dad4ae39dde6: No bootstrap required, opened a new log
I20260808 08:25:33.610585 4439 ts_tablet_manager.cc:1397] T 96e4a02ff77f47c9b543e15d0860c959 P 417a70ae534e48e580b4dad4ae39dde6: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260808 08:25:33.611073 4439 raft_consensus.cc:348] T 96e4a02ff77f47c9b543e15d0860c959 P 417a70ae534e48e580b4dad4ae39dde6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } }
I20260808 08:25:33.611173 4439 raft_consensus.cc:374] T 96e4a02ff77f47c9b543e15d0860c959 P 417a70ae534e48e580b4dad4ae39dde6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:33.611195 4439 raft_consensus.cc:729] T 96e4a02ff77f47c9b543e15d0860c959 P 417a70ae534e48e580b4dad4ae39dde6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 417a70ae534e48e580b4dad4ae39dde6, State: Initialized, Role: FOLLOWER
I20260808 08:25:33.611291 4439 consensus_queue.cc:260] T 96e4a02ff77f47c9b543e15d0860c959 P 417a70ae534e48e580b4dad4ae39dde6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } }
I20260808 08:25:33.611333 4438 tablet_bootstrap.cc:492] T 96e4a02ff77f47c9b543e15d0860c959 P bc0b21918b524c05aeada9beda0f7b75: No bootstrap required, opened a new log
I20260808 08:25:33.611410 4438 ts_tablet_manager.cc:1397] T 96e4a02ff77f47c9b543e15d0860c959 P bc0b21918b524c05aeada9beda0f7b75: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 08:25:33.611546 4439 ts_tablet_manager.cc:1428] T 96e4a02ff77f47c9b543e15d0860c959 P 417a70ae534e48e580b4dad4ae39dde6: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 08:25:33.611791 4438 raft_consensus.cc:348] T 96e4a02ff77f47c9b543e15d0860c959 P bc0b21918b524c05aeada9beda0f7b75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } }
I20260808 08:25:33.611901 4438 raft_consensus.cc:374] T 96e4a02ff77f47c9b543e15d0860c959 P bc0b21918b524c05aeada9beda0f7b75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:33.611933 4438 raft_consensus.cc:729] T 96e4a02ff77f47c9b543e15d0860c959 P bc0b21918b524c05aeada9beda0f7b75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc0b21918b524c05aeada9beda0f7b75, State: Initialized, Role: FOLLOWER
I20260808 08:25:33.612061 4438 consensus_queue.cc:260] T 96e4a02ff77f47c9b543e15d0860c959 P bc0b21918b524c05aeada9beda0f7b75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } }
I20260808 08:25:33.612300 4438 ts_tablet_manager.cc:1428] T 96e4a02ff77f47c9b543e15d0860c959 P bc0b21918b524c05aeada9beda0f7b75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 08:25:33.683825 4448 raft_consensus.cc:482] T 96e4a02ff77f47c9b543e15d0860c959 P bc0b21918b524c05aeada9beda0f7b75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 08:25:33.683955 4448 raft_consensus.cc:504] T 96e4a02ff77f47c9b543e15d0860c959 P bc0b21918b524c05aeada9beda0f7b75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } }
I20260808 08:25:33.684368 4448 leader_election.cc:288] T 96e4a02ff77f47c9b543e15d0860c959 P bc0b21918b524c05aeada9beda0f7b75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 417a70ae534e48e580b4dad4ae39dde6 (127.22.235.194:34595), a89f34ab8de74b31b499851c051728cc (127.22.235.195:45553)
I20260808 08:25:33.684831 4075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96e4a02ff77f47c9b543e15d0860c959" candidate_uuid: "bc0b21918b524c05aeada9beda0f7b75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "417a70ae534e48e580b4dad4ae39dde6" is_pre_election: true
I20260808 08:25:33.684831 4311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96e4a02ff77f47c9b543e15d0860c959" candidate_uuid: "bc0b21918b524c05aeada9beda0f7b75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a89f34ab8de74b31b499851c051728cc" is_pre_election: true
I20260808 08:25:33.684985 4075 raft_consensus.cc:2454] T 96e4a02ff77f47c9b543e15d0860c959 P 417a70ae534e48e580b4dad4ae39dde6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc0b21918b524c05aeada9beda0f7b75 in term 0.
I20260808 08:25:33.684985 4311 raft_consensus.cc:2454] T 96e4a02ff77f47c9b543e15d0860c959 P a89f34ab8de74b31b499851c051728cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc0b21918b524c05aeada9beda0f7b75 in term 0.
I20260808 08:25:33.685339 3699 leader_election.cc:302] T 96e4a02ff77f47c9b543e15d0860c959 P bc0b21918b524c05aeada9beda0f7b75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 417a70ae534e48e580b4dad4ae39dde6, a89f34ab8de74b31b499851c051728cc, bc0b21918b524c05aeada9beda0f7b75; no voters:
I20260808 08:25:33.685551 4448 raft_consensus.cc:2781] T 96e4a02ff77f47c9b543e15d0860c959 P bc0b21918b524c05aeada9beda0f7b75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 08:25:33.685621 4448 raft_consensus.cc:482] T 96e4a02ff77f47c9b543e15d0860c959 P bc0b21918b524c05aeada9beda0f7b75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 08:25:33.685667 4448 raft_consensus.cc:3037] T 96e4a02ff77f47c9b543e15d0860c959 P bc0b21918b524c05aeada9beda0f7b75 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:33.686707 4448 raft_consensus.cc:504] T 96e4a02ff77f47c9b543e15d0860c959 P bc0b21918b524c05aeada9beda0f7b75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } }
I20260808 08:25:33.687045 4448 leader_election.cc:288] T 96e4a02ff77f47c9b543e15d0860c959 P bc0b21918b524c05aeada9beda0f7b75 [CANDIDATE]: Term 1 election: Requested vote from peers 417a70ae534e48e580b4dad4ae39dde6 (127.22.235.194:34595), a89f34ab8de74b31b499851c051728cc (127.22.235.195:45553)
I20260808 08:25:33.687408 4311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96e4a02ff77f47c9b543e15d0860c959" candidate_uuid: "bc0b21918b524c05aeada9beda0f7b75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a89f34ab8de74b31b499851c051728cc"
I20260808 08:25:33.687422 4075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96e4a02ff77f47c9b543e15d0860c959" candidate_uuid: "bc0b21918b524c05aeada9beda0f7b75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "417a70ae534e48e580b4dad4ae39dde6"
I20260808 08:25:33.687526 4311 raft_consensus.cc:3037] T 96e4a02ff77f47c9b543e15d0860c959 P a89f34ab8de74b31b499851c051728cc [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:33.687546 4075 raft_consensus.cc:3037] T 96e4a02ff77f47c9b543e15d0860c959 P 417a70ae534e48e580b4dad4ae39dde6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:33.688419 4311 raft_consensus.cc:2454] T 96e4a02ff77f47c9b543e15d0860c959 P a89f34ab8de74b31b499851c051728cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc0b21918b524c05aeada9beda0f7b75 in term 1.
I20260808 08:25:33.688735 3699 leader_election.cc:302] T 96e4a02ff77f47c9b543e15d0860c959 P bc0b21918b524c05aeada9beda0f7b75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a89f34ab8de74b31b499851c051728cc, bc0b21918b524c05aeada9beda0f7b75; no voters:
I20260808 08:25:33.688774 4075 raft_consensus.cc:2454] T 96e4a02ff77f47c9b543e15d0860c959 P 417a70ae534e48e580b4dad4ae39dde6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc0b21918b524c05aeada9beda0f7b75 in term 1.
I20260808 08:25:33.689097 4448 raft_consensus.cc:2781] T 96e4a02ff77f47c9b543e15d0860c959 P bc0b21918b524c05aeada9beda0f7b75 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:25:33.689190 4448 raft_consensus.cc:686] T 96e4a02ff77f47c9b543e15d0860c959 P bc0b21918b524c05aeada9beda0f7b75 [term 1 LEADER]: Becoming Leader. State: Replica: bc0b21918b524c05aeada9beda0f7b75, State: Running, Role: LEADER
I20260808 08:25:33.689333 4448 consensus_queue.cc:237] T 96e4a02ff77f47c9b543e15d0860c959 P bc0b21918b524c05aeada9beda0f7b75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } }
I20260808 08:25:33.690981 3545 catalog_manager.cc:5495] T 96e4a02ff77f47c9b543e15d0860c959 P bc0b21918b524c05aeada9beda0f7b75 reported cstate change: term changed from 0 to 1, leader changed from <none> to bc0b21918b524c05aeada9beda0f7b75 (127.22.235.193). New cstate: current_term: 1 leader_uuid: "bc0b21918b524c05aeada9beda0f7b75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc0b21918b524c05aeada9beda0f7b75" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 46367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 } health_report { overall_health: UNKNOWN } } }
I20260808 08:25:33.725441 4528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:33.722559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:46367 (local address 127.22.235.193:46123)
0808 08:25:33.722673 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:33.722687 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:33.722767 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:33.722967 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:33.722970 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:33.722976 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 08:25:33.723091 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:33.723098 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:33.723778 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:33.723781 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:33.724358 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:33.724366 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:33.724821 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:33.724824 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:33.724930 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:33.724952 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 08:25:33.725186 (+ 234us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 08:25:33.725192 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 08:25:33.725249 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 08:25:33.725319 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":19}
I20260808 08:25:33.725441 4433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:33.722616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:46123 (local address 127.22.235.193:46367)
0808 08:25:33.722705 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:33.722709 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:33.722723 (+ 14us) server_negotiation.cc:407] Connection header received
0808 08:25:33.722850 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:33.722853 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:33.722894 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:33.722952 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:25:33.723176 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:33.723679 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:33.724462 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:33.724729 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:33.724815 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:33.725053 (+ 238us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:25:33.725122 (+ 69us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 08:25:33.725175 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:33.725256 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 08:25:33.725297 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":39}
I20260808 08:25:33.731670 4311 raft_consensus.cc:1261] T 96e4a02ff77f47c9b543e15d0860c959 P a89f34ab8de74b31b499851c051728cc [term 1 FOLLOWER]: Refusing update from remote peer bc0b21918b524c05aeada9beda0f7b75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:25:33.731673 4075 raft_consensus.cc:1261] T 96e4a02ff77f47c9b543e15d0860c959 P 417a70ae534e48e580b4dad4ae39dde6 [term 1 FOLLOWER]: Refusing update from remote peer bc0b21918b524c05aeada9beda0f7b75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:25:33.732038 4448 consensus_queue.cc:1035] T 96e4a02ff77f47c9b543e15d0860c959 P bc0b21918b524c05aeada9beda0f7b75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a89f34ab8de74b31b499851c051728cc" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:25:33.732190 4522 consensus_queue.cc:1035] T 96e4a02ff77f47c9b543e15d0860c959 P bc0b21918b524c05aeada9beda0f7b75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "417a70ae534e48e580b4dad4ae39dde6" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 34595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:25:33.764674 3545 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:25:33.764832 3545 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:25:33.766748 3545 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a626e0cf38834d9992e58716fe7ae506: Sending DeleteTablet for 3 replicas of tablet 96e4a02ff77f47c9b543e15d0860c959
I20260808 08:25:33.767307 4272 tablet_service.cc:1514] Processing DeleteTablet for tablet 96e4a02ff77f47c9b543e15d0860c959 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:25:33 UTC) from {username='oryx', principal='oryx/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:36776
I20260808 08:25:33.767318 4026 tablet_service.cc:1514] Processing DeleteTablet for tablet 96e4a02ff77f47c9b543e15d0860c959 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:25:33 UTC) from {username='oryx', principal='oryx/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:55482
I20260808 08:25:33.767477 4473 tablet_replica.cc:330] T 96e4a02ff77f47c9b543e15d0860c959 P a89f34ab8de74b31b499851c051728cc: stopping tablet replica
I20260808 08:25:33.767473 4530 tablet_replica.cc:330] T 96e4a02ff77f47c9b543e15d0860c959 P 417a70ae534e48e580b4dad4ae39dde6: stopping tablet replica
I20260808 08:25:33.767587 4530 raft_consensus.cc:2229] T 96e4a02ff77f47c9b543e15d0860c959 P 417a70ae534e48e580b4dad4ae39dde6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 08:25:33.767585 4473 raft_consensus.cc:2229] T 96e4a02ff77f47c9b543e15d0860c959 P a89f34ab8de74b31b499851c051728cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 08:25:33.767690 4530 raft_consensus.cc:2258] T 96e4a02ff77f47c9b543e15d0860c959 P 417a70ae534e48e580b4dad4ae39dde6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:25:33.767690 4473 raft_consensus.cc:2258] T 96e4a02ff77f47c9b543e15d0860c959 P a89f34ab8de74b31b499851c051728cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:25:33.767845 3784 tablet_service.cc:1514] Processing DeleteTablet for tablet 96e4a02ff77f47c9b543e15d0860c959 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:25:33 UTC) from {username='oryx', principal='oryx/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:36110
I20260808 08:25:33.768324 4530 ts_tablet_manager.cc:1905] T 96e4a02ff77f47c9b543e15d0860c959 P 417a70ae534e48e580b4dad4ae39dde6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:25:33.768369 4473 ts_tablet_manager.cc:1905] T 96e4a02ff77f47c9b543e15d0860c959 P a89f34ab8de74b31b499851c051728cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:25:33.770282 4473 ts_tablet_manager.cc:1918] T 96e4a02ff77f47c9b543e15d0860c959 P a89f34ab8de74b31b499851c051728cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 08:25:33.770345 4473 log.cc:1196] T 96e4a02ff77f47c9b543e15d0860c959 P a89f34ab8de74b31b499851c051728cc: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-2/wal/wals/96e4a02ff77f47c9b543e15d0860c959
I20260808 08:25:33.770323 4530 ts_tablet_manager.cc:1918] T 96e4a02ff77f47c9b543e15d0860c959 P 417a70ae534e48e580b4dad4ae39dde6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 08:25:33.770394 4530 log.cc:1196] T 96e4a02ff77f47c9b543e15d0860c959 P 417a70ae534e48e580b4dad4ae39dde6: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786177493204855-23471-0/minicluster-data/ts-1/wal/wals/96e4a02ff77f47c9b543e15d0860c959
I20260808 08:25:33.770555 4473 ts_tablet_manager.cc:1939] T 96e4a02ff77f47c9b543e15d0860c959 P a89f34ab8de74b31b499851c051728cc: Deleting consensus metadata
I20260808 08:25:33.770650 4530 ts_tablet_manager.cc:1939] T 96e4a02ff77f47c9b543e15d0860c959 P 417a70ae534e48e580b4dad4ae39dde6: Deleting consensus metadata
I20260808 08:25:33.771139 3520 catalog_manager.cc:4841] TS a89f34ab8de74b31b499851c051728cc (127.22.235.195:45553): tablet 96e4a02ff77f47c9b543e15d0860c959 (table test-table [id=bb4577859e6b425196d70954115a5125]) successfully deleted
I20260808 08:25:33.771248 4471 tablet_replica.cc:330] T 96e4a02ff77f47c9b543e15d0860c959 P bc0b21918b524c05aeada9beda0f7b75: stopping tablet replica
I20260808 08:25:33.771364 4471 raft_consensus.cc:2229] T 96e4a02ff77f47c9b543e15d0860c959 P bc0b21918b524c05aeada9beda0f7b75 [term 1 LEADER]: Raft consensus shutting down.
I20260808 08:25:33.771540 4471 raft_consensus.cc:2258] T 96e4a02ff77f47c9b543e15d0860c959 P bc0b21918b524c05aeada9beda0f7b75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:25:33.771694 3524 catalog_manager.cc:4841] TS 417a70ae534e48e580b4dad4ae39dde6 (127.22.235.194:34595): tablet 96e4a02ff77f47c9b543e15d0860c959 (table test-table [id=bb4577859e6b425196d70954115a5125]) successfully deleted
I20260808 08:25:33.772477 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 3673
I20260808 08:25:33.772585 4471 ts_tablet_manager.cc:1905] T 96e4a02ff77f47c9b543e15d0860c959 P bc0b21918b524c05aeada9beda0f7b75: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 08:25:33.787529 3522 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.235.193:46367 (error 108)
I20260808 08:25:33.787676 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 3919
I20260808 08:25:33.798065 4590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:33.794807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:46367 (local address 127.0.0.1:36128)
0808 08:25:33.797884 (+ 3077us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:33.797968 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.193:46367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2992,"thread_start_us":855,"threads_started":1}
W20260808 08:25:33.798247 3522 catalog_manager.cc:4584] TS bc0b21918b524c05aeada9beda0f7b75 (127.22.235.193:46367): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 96e4a02ff77f47c9b543e15d0860c959: Network error: Client connection negotiation failed: client connection to 127.22.235.193:46367: connect: Connection refused (error 111)
I20260808 08:25:33.803876 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 4164
I20260808 08:25:33.820076 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 3492
2026-08-08T08:25:33Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3252 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786177493204855-23471-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 08:25:33 dist-test-slave-f7th krb5kdc[4597](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 08:25:33 dist-test-slave-f7th krb5kdc[4597](info): set up 2 sockets
Aug 08 08:25:33 dist-test-slave-f7th krb5kdc[4597](info): commencing operation
krb5kdc: starting...
W20260808 08:25:34.987383 23471 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Aug 08 08:25:35 dist-test-slave-f7th krb5kdc[4597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177535, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T08:25:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:25:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.235.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
I20260808 08:25:35.104529 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:40673
--webserver_interface=127.22.235.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:45959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:40673
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:25:35.215212 4616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:35.215476 4616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:35.215528 4616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:35.219445 4616 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:25:35.219501 4616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:35.219527 4616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:35.219547 4616 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:25:35.219568 4616 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:25:35.224674 4616 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:45959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:40673
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:40673
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.235.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:35.225649 4616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:35.226637 4616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:35.234963 4625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:35.235013 4616 server_base.cc:1034] running on GCE node
W20260808 08:25:35.234947 4632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:35.234925 4626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:35.235699 4616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:35.236764 4616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:35.237916 4616 hybrid_clock.cc:648] HybridClock initialized: now 1786177535237889 us; error 39 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-08T08:25:35Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1420 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 08:25:35 dist-test-slave-f7th krb5kdc[4640](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 08:25:35 dist-test-slave-f7th krb5kdc[4640](info): set up 2 sockets
Aug 08 08:25:35 dist-test-slave-f7th krb5kdc[4640](info): commencing operation
krb5kdc: starting...
W20260808 08:25:36.406807 23471 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Aug 08 08:25:36 dist-test-slave-f7th krb5kdc[4640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177536, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T08:25:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:25:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.235.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
I20260808 08:25:36.522971 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:42715
--webserver_interface=127.22.235.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:46085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:42715
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:25:36.633005 4666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:36.633270 4666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:36.633309 4666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:36.637216 4666 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:25:36.637274 4666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:36.637292 4666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:36.637306 4666 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:25:36.637324 4666 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:25:36.642570 4666 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:46085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:42715
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:42715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.235.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:36.643564 4666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:36.644546 4666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:36.653127 4676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:36.653254 4666 server_base.cc:1034] running on GCE node
W20260808 08:25:36.653162 4675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:36.653445 4680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:36.653993 4666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:36.655112 4666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:36.656271 4666 hybrid_clock.cc:648] HybridClock initialized: now 1786177536656255 us; error 25 us; skew 500 ppm
Aug 08 08:25:36 dist-test-slave-f7th krb5kdc[4640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177536, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:36.658927 4666 init.cc:377] Logged in from keytab as kudu/127.22.235.254@KRBTEST.COM (short username kudu)
I20260808 08:25:36.659744 4666 webserver.cc:460] Webserver started at http://127.22.235.254:34933/ using document root <none> and password file <none>
I20260808 08:25:36.660264 4666 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:36.660319 4666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:36.660491 4666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:36.662515 4666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "3eaaf873a9b74e3d83276e19b8d7c150"
format_stamp: "Formatted at 2026-08-08 08:25:36 on dist-test-slave-f7th"
server_key: "a8fb30d4cdd3769293fb23d0dad366b9"
server_key_iv: "826d2e2f2b0fbbd10e0bb1e2db6db308"
server_key_version: "encryptionkey@0"
I20260808 08:25:36.663028 4666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "3eaaf873a9b74e3d83276e19b8d7c150"
format_stamp: "Formatted at 2026-08-08 08:25:36 on dist-test-slave-f7th"
server_key: "a8fb30d4cdd3769293fb23d0dad366b9"
server_key_iv: "826d2e2f2b0fbbd10e0bb1e2db6db308"
server_key_version: "encryptionkey@0"
I20260808 08:25:36.666532 4666 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 08:25:36.668789 4691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:36.669916 4666 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:25:36.670028 4666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "3eaaf873a9b74e3d83276e19b8d7c150"
format_stamp: "Formatted at 2026-08-08 08:25:36 on dist-test-slave-f7th"
server_key: "a8fb30d4cdd3769293fb23d0dad366b9"
server_key_iv: "826d2e2f2b0fbbd10e0bb1e2db6db308"
server_key_version: "encryptionkey@0"
I20260808 08:25:36.670109 4666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:36.689530 4666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:36.692116 4666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:36.692274 4666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:36.698804 4666 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.254:42715
I20260808 08:25:36.698861 4791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.254:42715 every 8 connection(s)
I20260808 08:25:36.699738 4666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:25:36.702455 4794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:36.707764 4794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3eaaf873a9b74e3d83276e19b8d7c150: Bootstrap starting.
I20260808 08:25:36.709151 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 4666
I20260808 08:25:36.709275 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:25:36.709575 23471 external_mini_cluster.cc:1428] Setting key 82d11afee7f95cb8b9d109faf0f94c93
I20260808 08:25:36.710433 4794 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3eaaf873a9b74e3d83276e19b8d7c150: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:36.711143 4794 log.cc:824] T 00000000000000000000000000000000 P 3eaaf873a9b74e3d83276e19b8d7c150: Log is configured to *not* fsync() on all Append() calls
I20260808 08:25:36.713157 4794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3eaaf873a9b74e3d83276e19b8d7c150: No bootstrap required, opened a new log
I20260808 08:25:36.717063 4794 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3eaaf873a9b74e3d83276e19b8d7c150 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eaaf873a9b74e3d83276e19b8d7c150" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 42715 } }
I20260808 08:25:36.717335 4794 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3eaaf873a9b74e3d83276e19b8d7c150 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:36.717394 4794 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3eaaf873a9b74e3d83276e19b8d7c150 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eaaf873a9b74e3d83276e19b8d7c150, State: Initialized, Role: FOLLOWER
I20260808 08:25:36.717852 4794 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3eaaf873a9b74e3d83276e19b8d7c150 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eaaf873a9b74e3d83276e19b8d7c150" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 42715 } }
I20260808 08:25:36.717978 4794 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3eaaf873a9b74e3d83276e19b8d7c150 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:25:36.718045 4794 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3eaaf873a9b74e3d83276e19b8d7c150 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:25:36.718140 4794 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3eaaf873a9b74e3d83276e19b8d7c150 [term 0 FOLLOWER]: Advancing to term 1
Aug 08 08:25:36 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177536, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:25:36.719051 4794 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3eaaf873a9b74e3d83276e19b8d7c150 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eaaf873a9b74e3d83276e19b8d7c150" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 42715 } }
I20260808 08:25:36.719393 4794 leader_election.cc:302] T 00000000000000000000000000000000 P 3eaaf873a9b74e3d83276e19b8d7c150 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3eaaf873a9b74e3d83276e19b8d7c150; no voters:
I20260808 08:25:36.719729 4794 leader_election.cc:288] T 00000000000000000000000000000000 P 3eaaf873a9b74e3d83276e19b8d7c150 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:25:36.719996 4804 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3eaaf873a9b74e3d83276e19b8d7c150 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:25:36.720188 4804 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3eaaf873a9b74e3d83276e19b8d7c150 [term 1 LEADER]: Becoming Leader. State: Replica: 3eaaf873a9b74e3d83276e19b8d7c150, State: Running, Role: LEADER
I20260808 08:25:36.720546 4804 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3eaaf873a9b74e3d83276e19b8d7c150 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eaaf873a9b74e3d83276e19b8d7c150" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 42715 } }
I20260808 08:25:36.721136 4794 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3eaaf873a9b74e3d83276e19b8d7c150 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:25:36.722324 4807 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3eaaf873a9b74e3d83276e19b8d7c150 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3eaaf873a9b74e3d83276e19b8d7c150. Latest consensus state: current_term: 1 leader_uuid: "3eaaf873a9b74e3d83276e19b8d7c150" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eaaf873a9b74e3d83276e19b8d7c150" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 42715 } } }
I20260808 08:25:36.722470 4807 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3eaaf873a9b74e3d83276e19b8d7c150 [sys.catalog]: This master's current role is: LEADER
I20260808 08:25:36.722325 4808 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3eaaf873a9b74e3d83276e19b8d7c150 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3eaaf873a9b74e3d83276e19b8d7c150" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eaaf873a9b74e3d83276e19b8d7c150" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 42715 } } }
I20260808 08:25:36.722680 4808 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3eaaf873a9b74e3d83276e19b8d7c150 [sys.catalog]: This master's current role is: LEADER
I20260808 08:25:36.723003 4821 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:25:36.724582 4802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:36.710700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48846 (local address 127.22.235.254:42715)
0808 08:25:36.711137 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:36.711153 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:36.711188 (+ 35us) server_negotiation.cc:407] Connection header received
0808 08:25:36.711925 (+ 737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:36.711946 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:36.712263 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:36.712528 (+ 265us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:36.713440 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:36.714401 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:36.715569 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:36.716012 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:36.716121 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:36.718865 (+ 2744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:36.718872 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:36.718888 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:36.718920 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:36.720890 (+ 1970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:36.721408 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:36.721415 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:36.721420 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:36.721567 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:36.721900 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:36.721903 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:36.721905 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:36.722419 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:36.723367 (+ 948us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:36.723520 (+ 153us) server_negotiation.cc:299] Negotiation successful
0808 08:25:36.723751 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":126,"threads_started":1}
I20260808 08:25:36.726294 4821 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:25:36.731577 4821 catalog_manager.cc:1347] Generated new cluster ID: f69fef0bdad04b23b6271a7b0831ddac
I20260808 08:25:36.731642 4821 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:25:36.742888 4821 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:25:36.744060 4821 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:25:36.752249 4821 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3eaaf873a9b74e3d83276e19b8d7c150: Generated new TSK 0
I20260808 08:25:36.753011 4821 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.235.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab.
Entry for principal kudu/127.22.235.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab.
I20260808 08:25:36.782084 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.193:0
--local_ip_for_outbound_sockets=127.22.235.193
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:42715
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab
--principal=kudu/127.22.235.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:46085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:25:36.894413 4838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:36.894651 4838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:36.894691 4838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:36.898638 4838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:36.898696 4838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:36.898761 4838 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.193
I20260808 08:25:36.903844 4838 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:46085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab
--principal=kudu/127.22.235.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:42715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4838
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.193
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:36.904925 4838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:36.905934 4838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:36.915221 4846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:36.915221 4847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:36.915187 4849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:36.915484 4838 server_base.cc:1034] running on GCE node
I20260808 08:25:36.915866 4838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:36.916512 4838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:36.917697 4838 hybrid_clock.cc:648] HybridClock initialized: now 1786177536917669 us; error 37 us; skew 500 ppm
Aug 08 08:25:36 dist-test-slave-f7th krb5kdc[4640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177536, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:36.920266 4838 init.cc:377] Logged in from keytab as kudu/127.22.235.193@KRBTEST.COM (short username kudu)
I20260808 08:25:36.921172 4838 webserver.cc:460] Webserver started at http://127.22.235.193:41229/ using document root <none> and password file <none>
I20260808 08:25:36.921720 4838 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:36.921833 4838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:36.922035 4838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:36.924039 4838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-0/data/instance:
uuid: "2340af5975234199b584739caef0460e"
format_stamp: "Formatted at 2026-08-08 08:25:36 on dist-test-slave-f7th"
server_key: "7ad616f248a786b94ba7a6961ffcaf60"
server_key_iv: "60d0771e33e00b168814a614af7009f6"
server_key_version: "encryptionkey@0"
I20260808 08:25:36.924521 4838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance:
uuid: "2340af5975234199b584739caef0460e"
format_stamp: "Formatted at 2026-08-08 08:25:36 on dist-test-slave-f7th"
server_key: "7ad616f248a786b94ba7a6961ffcaf60"
server_key_iv: "60d0771e33e00b168814a614af7009f6"
server_key_version: "encryptionkey@0"
I20260808 08:25:36.927968 4838 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 08:25:36.930266 4863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:36.931391 4838 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 08:25:36.931497 4838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-0/wal
uuid: "2340af5975234199b584739caef0460e"
format_stamp: "Formatted at 2026-08-08 08:25:36 on dist-test-slave-f7th"
server_key: "7ad616f248a786b94ba7a6961ffcaf60"
server_key_iv: "60d0771e33e00b168814a614af7009f6"
server_key_version: "encryptionkey@0"
I20260808 08:25:36.931578 4838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:36.990118 4838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:36.992888 4838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:36.993054 4838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:36.993605 4838 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:25:36.994422 4838 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:25:36.994468 4838 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:36.994508 4838 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:25:36.994526 4838 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:37.003036 4838 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.193:45529
I20260808 08:25:37.003053 5066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.193:45529 every 8 connection(s)
I20260808 08:25:37.003968 4838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
I20260808 08:25:37.009083 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 4838
I20260808 08:25:37.009209 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance
I20260808 08:25:37.009486 23471 external_mini_cluster.cc:1428] Setting key 50fc3cd8628dac93618d8cbc35d6854a
Aug 08 08:25:37 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177536, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:25:37.018168 4802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:37.005807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:47485 (local address 127.22.235.254:42715)
0808 08:25:37.005948 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:37.005952 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:37.006798 (+ 846us) server_negotiation.cc:407] Connection header received
0808 08:25:37.008121 (+ 1323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:37.008124 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:37.008184 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:37.008257 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:37.009513 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:37.010140 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:37.011582 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:37.011808 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:37.011889 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:37.014564 (+ 2675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:37.014568 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:37.014571 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:37.014598 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:37.016130 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:37.016633 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:37.016638 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:37.016640 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:37.016736 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:37.017042 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:37.017045 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:37.017047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:37.017275 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:37.017381 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:37.017843 (+ 462us) server_negotiation.cc:299] Negotiation successful
0808 08:25:37.017981 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 08:25:37.018890 5069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:37.006182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.193:47485)
0808 08:25:37.006638 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:37.006680 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:37.007909 (+ 1229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:37.008443 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:37.008450 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:37.008859 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:37.009374 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:37.009390 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:37.010270 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:37.010273 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:37.011451 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:37.011475 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:37.011956 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:37.011960 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:37.012214 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:37.012865 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:37.012889 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:37.014394 (+ 1505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:37.016240 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:37.016245 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:37.016257 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:37.016515 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:37.016821 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:37.016823 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:37.016825 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:37.016957 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:37.017365 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:37.017369 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:37.017616 (+ 247us) client_negotiation.cc:769] Sending connection context
0808 08:25:37.017769 (+ 153us) client_negotiation.cc:240] Negotiation successful
0808 08:25:37.018023 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":121,"threads_started":1}
WARNING: no policy specified for kudu/127.22.235.194@KRBTEST.COM; defaulting to no policy
I20260808 08:25:37.020061 5067 heartbeater.cc:347] Connected to a master server at 127.22.235.254:42715
I20260808 08:25:37.020304 5067 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:37.020843 5067 heartbeater.cc:511] Master 127.22.235.254:42715 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.194@KRBTEST.COM" created.
I20260808 08:25:37.022655 4729 ts_manager.cc:194] Registered new tserver with Master: 2340af5975234199b584739caef0460e (127.22.235.193:45529)
I20260808 08:25:37.024138 4729 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.193@KRBTEST.COM'} at 127.22.235.193:47485
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab.
Entry for principal kudu/127.22.235.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab.
I20260808 08:25:37.037691 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.194:0
--local_ip_for_outbound_sockets=127.22.235.194
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:42715
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab
--principal=kudu/127.22.235.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:46085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:25:37.148556 5076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:37.148842 5076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:37.148890 5076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:37.152814 5076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:37.152868 5076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:37.152933 5076 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.194
I20260808 08:25:37.158052 5076 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:46085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab
--principal=kudu/127.22.235.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:42715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5076
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.194
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:37.159019 5076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:37.159994 5076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 08:25:37.169315 5076 server_base.cc:1034] running on GCE node
W20260808 08:25:37.169277 5085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:37.169281 5081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:37.169256 5083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:37.169874 5076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:37.170590 5076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:37.171749 5076 hybrid_clock.cc:648] HybridClock initialized: now 1786177537171730 us; error 29 us; skew 500 ppm
Aug 08 08:25:37 dist-test-slave-f7th krb5kdc[4640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177537, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:37.174494 5076 init.cc:377] Logged in from keytab as kudu/127.22.235.194@KRBTEST.COM (short username kudu)
I20260808 08:25:37.175271 5076 webserver.cc:460] Webserver started at http://127.22.235.194:36965/ using document root <none> and password file <none>
I20260808 08:25:37.175776 5076 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:37.175833 5076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:37.175995 5076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:37.177953 5076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-1/data/instance:
uuid: "3968a62d7d4141c9a80e559af53d5e29"
format_stamp: "Formatted at 2026-08-08 08:25:37 on dist-test-slave-f7th"
server_key: "795074592c7b659e0b038dbcee502e37"
server_key_iv: "7895694babce2fe9489d4e801e194469"
server_key_version: "encryptionkey@0"
I20260808 08:25:37.178431 5076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance:
uuid: "3968a62d7d4141c9a80e559af53d5e29"
format_stamp: "Formatted at 2026-08-08 08:25:37 on dist-test-slave-f7th"
server_key: "795074592c7b659e0b038dbcee502e37"
server_key_iv: "7895694babce2fe9489d4e801e194469"
server_key_version: "encryptionkey@0"
I20260808 08:25:37.181926 5076 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 08:25:37.184219 5098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:37.185357 5076 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:25:37.185456 5076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-1/wal
uuid: "3968a62d7d4141c9a80e559af53d5e29"
format_stamp: "Formatted at 2026-08-08 08:25:37 on dist-test-slave-f7th"
server_key: "795074592c7b659e0b038dbcee502e37"
server_key_iv: "7895694babce2fe9489d4e801e194469"
server_key_version: "encryptionkey@0"
I20260808 08:25:37.185544 5076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:37.209671 5076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:37.212282 5076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:37.212430 5076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:37.212970 5076 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:25:37.213770 5076 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:25:37.213819 5076 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:37.213857 5076 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:25:37.213877 5076 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:37.222335 5076 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.194:37991
I20260808 08:25:37.222360 5287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.194:37991 every 8 connection(s)
I20260808 08:25:37.223234 5076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
I20260808 08:25:37.223773 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 5076
I20260808 08:25:37.223878 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance
I20260808 08:25:37.224136 23471 external_mini_cluster.cc:1428] Setting key 537a5e7306514fb42129a796c47a041d
Aug 08 08:25:37 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177537, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.194@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.235.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.195@KRBTEST.COM" created.
I20260808 08:25:37.237695 4802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:37.226467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:48033 (local address 127.22.235.254:42715)
0808 08:25:37.226612 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:37.226617 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:37.226634 (+ 17us) server_negotiation.cc:407] Connection header received
0808 08:25:37.227551 (+ 917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:37.227555 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:37.227604 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:37.227668 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:37.229150 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:37.229667 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:37.231284 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:37.231509 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:37.231584 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:37.234439 (+ 2855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:37.234442 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:37.234444 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:37.234467 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:37.235810 (+ 1343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:37.236274 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:37.236278 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:37.236279 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:37.236355 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:37.236667 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:37.236670 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:37.236672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:37.236845 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:37.236931 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:37.237404 (+ 473us) server_negotiation.cc:299] Negotiation successful
0808 08:25:37.237534 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 08:25:37.238513 5293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:37.225425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.194:48033)
0808 08:25:37.225948 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:37.226010 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:37.227312 (+ 1302us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:37.227852 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:37.227861 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:37.228336 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:37.229019 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:37.229040 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:37.229789 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:37.229792 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:37.231176 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:37.231188 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:37.231657 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:37.231661 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:37.231860 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:37.232538 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:37.232576 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:37.234280 (+ 1704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:37.235911 (+ 1631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:37.235917 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:37.235931 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:37.236180 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:37.236437 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:37.236440 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:37.236442 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:37.236573 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:37.236942 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:37.236949 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:37.237180 (+ 231us) client_negotiation.cc:769] Sending connection context
0808 08:25:37.237343 (+ 163us) client_negotiation.cc:240] Negotiation successful
0808 08:25:37.237647 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":141,"threads_started":1}
I20260808 08:25:37.239687 5288 heartbeater.cc:347] Connected to a master server at 127.22.235.254:42715
I20260808 08:25:37.239903 5288 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:37.240361 5288 heartbeater.cc:511] Master 127.22.235.254:42715 requested a full tablet report, sending...
I20260808 08:25:37.241477 4729 ts_manager.cc:194] Registered new tserver with Master: 3968a62d7d4141c9a80e559af53d5e29 (127.22.235.194:37991)
I20260808 08:25:37.242240 4729 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.194@KRBTEST.COM'} at 127.22.235.194:48033
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab.
Entry for principal kudu/127.22.235.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab.
I20260808 08:25:37.251906 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.195:0
--local_ip_for_outbound_sockets=127.22.235.195
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:42715
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab
--principal=kudu/127.22.235.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:46085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:25:37.363516 5300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:37.363750 5300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:37.363790 5300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:37.367687 5300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:37.367738 5300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:37.367790 5300 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.195
I20260808 08:25:37.372763 5300 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:46085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab
--principal=kudu/127.22.235.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:42715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.195
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:37.373744 5300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:37.374784 5300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:37.383931 5308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:37.384055 5313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:37.383950 5310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:37.384153 5300 server_base.cc:1034] running on GCE node
I20260808 08:25:37.384590 5300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:37.385239 5300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:37.386391 5300 hybrid_clock.cc:648] HybridClock initialized: now 1786177537386368 us; error 34 us; skew 500 ppm
Aug 08 08:25:37 dist-test-slave-f7th krb5kdc[4640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177537, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:37.389053 5300 init.cc:377] Logged in from keytab as kudu/127.22.235.195@KRBTEST.COM (short username kudu)
I20260808 08:25:37.389890 5300 webserver.cc:460] Webserver started at http://127.22.235.195:36991/ using document root <none> and password file <none>
I20260808 08:25:37.390422 5300 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:37.390487 5300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:37.390676 5300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:37.392607 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-2/data/instance:
uuid: "fbc3be67bdcb41099cd16c94f40c7e58"
format_stamp: "Formatted at 2026-08-08 08:25:37 on dist-test-slave-f7th"
server_key: "b7199656bd3364832b6763c7fed16a69"
server_key_iv: "b3152ec58833ff057db65c9d76881f52"
server_key_version: "encryptionkey@0"
I20260808 08:25:37.393168 5300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance:
uuid: "fbc3be67bdcb41099cd16c94f40c7e58"
format_stamp: "Formatted at 2026-08-08 08:25:37 on dist-test-slave-f7th"
server_key: "b7199656bd3364832b6763c7fed16a69"
server_key_iv: "b3152ec58833ff057db65c9d76881f52"
server_key_version: "encryptionkey@0"
I20260808 08:25:37.396564 5300 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 08:25:37.398836 5325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:37.399940 5300 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 08:25:37.400053 5300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-2/wal
uuid: "fbc3be67bdcb41099cd16c94f40c7e58"
format_stamp: "Formatted at 2026-08-08 08:25:37 on dist-test-slave-f7th"
server_key: "b7199656bd3364832b6763c7fed16a69"
server_key_iv: "b3152ec58833ff057db65c9d76881f52"
server_key_version: "encryptionkey@0"
I20260808 08:25:37.400141 5300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:37.411096 5300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:37.413617 5300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:37.413776 5300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:37.414307 5300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:25:37.415097 5300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:25:37.415144 5300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:37.415196 5300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:25:37.415225 5300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:37.423723 5300 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.195:45013
I20260808 08:25:37.423775 5512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.195:45013 every 8 connection(s)
I20260808 08:25:37.424559 5300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
I20260808 08:25:37.428275 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 5300
I20260808 08:25:37.428392 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance
I20260808 08:25:37.428742 23471 external_mini_cluster.cc:1428] Setting key 9d33bc7c97194ea9014d49edd4fb4043
Aug 08 08:25:37 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177537, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.195@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:25:37.437183 4802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:37.426254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:55055 (local address 127.22.235.254:42715)
0808 08:25:37.426391 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:37.426395 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:37.427161 (+ 766us) server_negotiation.cc:407] Connection header received
0808 08:25:37.428441 (+ 1280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:37.428445 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:37.428505 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:37.428586 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:37.429883 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:37.430460 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:37.431612 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:37.431831 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:37.431906 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:37.434142 (+ 2236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:37.434146 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:37.434148 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:37.434174 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:37.435412 (+ 1238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:37.435873 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:37.435877 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:37.435878 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:37.435956 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:37.436250 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:37.436252 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:37.436253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:37.436406 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:37.436474 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:37.436964 (+ 490us) server_negotiation.cc:299] Negotiation successful
0808 08:25:37.437064 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 08:25:37.437903 5520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:37.426594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.195:55055)
0808 08:25:37.427022 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:37.427063 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:37.428227 (+ 1164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:37.428770 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:37.428777 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:37.429133 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:37.429725 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:37.429743 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:37.430595 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:37.430599 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:37.431511 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:37.431521 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:37.431941 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:37.431944 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:37.432083 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:37.432582 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:37.432597 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:37.433987 (+ 1390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:37.435517 (+ 1530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:37.435522 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:37.435534 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:37.435776 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:37.436040 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:37.436042 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:37.436044 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:37.436184 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:37.436483 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:37.436487 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:37.436752 (+ 265us) client_negotiation.cc:769] Sending connection context
0808 08:25:37.436905 (+ 153us) client_negotiation.cc:240] Negotiation successful
0808 08:25:37.437146 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":119,"threads_started":1}
I20260808 08:25:37.438915 5515 heartbeater.cc:347] Connected to a master server at 127.22.235.254:42715
I20260808 08:25:37.439137 5515 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:37.439603 5515 heartbeater.cc:511] Master 127.22.235.254:42715 requested a full tablet report, sending...
I20260808 08:25:37.440560 4729 ts_manager.cc:194] Registered new tserver with Master: fbc3be67bdcb41099cd16c94f40c7e58 (127.22.235.195:45013)
I20260808 08:25:37.441161 4729 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.195@KRBTEST.COM'} at 127.22.235.195:55055
I20260808 08:25:37.442260 23471 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.22.235.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.235.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.235.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.193.keytab.
Entry for principal oryx/127.22.235.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.193.keytab.
WARNING: no policy specified for oryx/127.22.235.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.235.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.235.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.194.keytab.
Entry for principal oryx/127.22.235.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.194.keytab.
WARNING: no policy specified for oryx/127.22.235.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.235.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.235.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.195.keytab.
Entry for principal oryx/127.22.235.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.195.keytab.
I20260808 08:25:37.508975 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 4838
I20260808 08:25:37.546854 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 5076
I20260808 08:25:37.564961 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 5300
I20260808 08:25:37.573225 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 4666
I20260808 08:25:37.592025 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:42715
--webserver_interface=127.22.235.254
--webserver_port=34933
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:46085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:42715
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:25:37.703083 5530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:37.703325 5530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:37.703361 5530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:37.707206 5530 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:25:37.707252 5530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:37.707269 5530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:37.707288 5530 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:25:37.707304 5530 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:25:37.712320 5530 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:46085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:42715
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:42715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.235.254
--webserver_port=34933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5530
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:37.713270 5530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:37.714198 5530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:37.722389 5539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:37.722519 5530 server_base.cc:1034] running on GCE node
W20260808 08:25:37.722411 5544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:37.722393 5540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:37.723225 5530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:37.724254 5530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:37.725406 5530 hybrid_clock.cc:648] HybridClock initialized: now 1786177537725390 us; error 27 us; skew 500 ppm
Aug 08 08:25:37 dist-test-slave-f7th krb5kdc[4640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177537, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:37.728016 5530 init.cc:377] Logged in from keytab as kudu/127.22.235.254@KRBTEST.COM (short username kudu)
I20260808 08:25:37.728899 5530 webserver.cc:460] Webserver started at http://127.22.235.254:34933/ using document root <none> and password file <none>
I20260808 08:25:37.729410 5530 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:37.729485 5530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:37.733139 5530 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:25:37.735110 5559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:37.736245 5530 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:25:37.736351 5530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "3eaaf873a9b74e3d83276e19b8d7c150"
format_stamp: "Formatted at 2026-08-08 08:25:36 on dist-test-slave-f7th"
server_key: "a8fb30d4cdd3769293fb23d0dad366b9"
server_key_iv: "826d2e2f2b0fbbd10e0bb1e2db6db308"
server_key_version: "encryptionkey@0"
I20260808 08:25:37.736882 5530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:37.746930 5530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:37.749909 5530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:37.750083 5530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:37.756774 5530 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.254:42715
I20260808 08:25:37.756783 5641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.254:42715 every 8 connection(s)
I20260808 08:25:37.757798 5530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:25:37.761988 5642 sys_catalog.cc:263] Verifying existing consensus state
I20260808 08:25:37.763558 5642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3eaaf873a9b74e3d83276e19b8d7c150: Bootstrap starting.
I20260808 08:25:37.767656 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 5530
I20260808 08:25:37.768157 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.193:45529
--local_ip_for_outbound_sockets=127.22.235.193
--tserver_master_addrs=127.22.235.254:42715
--webserver_port=41229
--webserver_interface=127.22.235.193
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab
--principal=kudu/127.22.235.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:46085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.22.235.193
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 08:25:37.780757 5642 log.cc:824] T 00000000000000000000000000000000 P 3eaaf873a9b74e3d83276e19b8d7c150: Log is configured to *not* fsync() on all Append() calls
I20260808 08:25:37.787487 5642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3eaaf873a9b74e3d83276e19b8d7c150: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 08:25:37.787887 5642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3eaaf873a9b74e3d83276e19b8d7c150: Bootstrap complete.
I20260808 08:25:37.791884 5642 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3eaaf873a9b74e3d83276e19b8d7c150 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eaaf873a9b74e3d83276e19b8d7c150" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 42715 } }
I20260808 08:25:37.792276 5642 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3eaaf873a9b74e3d83276e19b8d7c150 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eaaf873a9b74e3d83276e19b8d7c150, State: Initialized, Role: FOLLOWER
I20260808 08:25:37.792776 5642 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3eaaf873a9b74e3d83276e19b8d7c150 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eaaf873a9b74e3d83276e19b8d7c150" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 42715 } }
I20260808 08:25:37.792896 5642 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3eaaf873a9b74e3d83276e19b8d7c150 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:25:37.792964 5642 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3eaaf873a9b74e3d83276e19b8d7c150 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:25:37.793064 5642 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3eaaf873a9b74e3d83276e19b8d7c150 [term 1 FOLLOWER]: Advancing to term 2
I20260808 08:25:37.794920 5642 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3eaaf873a9b74e3d83276e19b8d7c150 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eaaf873a9b74e3d83276e19b8d7c150" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 42715 } }
I20260808 08:25:37.795226 5642 leader_election.cc:302] T 00000000000000000000000000000000 P 3eaaf873a9b74e3d83276e19b8d7c150 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3eaaf873a9b74e3d83276e19b8d7c150; no voters:
I20260808 08:25:37.795522 5642 leader_election.cc:288] T 00000000000000000000000000000000 P 3eaaf873a9b74e3d83276e19b8d7c150 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 08:25:37.795619 5647 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3eaaf873a9b74e3d83276e19b8d7c150 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 08:25:37.795809 5647 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3eaaf873a9b74e3d83276e19b8d7c150 [term 2 LEADER]: Becoming Leader. State: Replica: 3eaaf873a9b74e3d83276e19b8d7c150, State: Running, Role: LEADER
I20260808 08:25:37.796103 5647 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3eaaf873a9b74e3d83276e19b8d7c150 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eaaf873a9b74e3d83276e19b8d7c150" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 42715 } }
I20260808 08:25:37.796787 5642 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3eaaf873a9b74e3d83276e19b8d7c150 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:25:37.797047 5650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3eaaf873a9b74e3d83276e19b8d7c150 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3eaaf873a9b74e3d83276e19b8d7c150. Latest consensus state: current_term: 2 leader_uuid: "3eaaf873a9b74e3d83276e19b8d7c150" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eaaf873a9b74e3d83276e19b8d7c150" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 42715 } } }
I20260808 08:25:37.797165 5650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3eaaf873a9b74e3d83276e19b8d7c150 [sys.catalog]: This master's current role is: LEADER
I20260808 08:25:37.797417 5649 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3eaaf873a9b74e3d83276e19b8d7c150 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "3eaaf873a9b74e3d83276e19b8d7c150" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eaaf873a9b74e3d83276e19b8d7c150" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 42715 } } }
I20260808 08:25:37.797519 5654 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:25:37.797528 5649 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3eaaf873a9b74e3d83276e19b8d7c150 [sys.catalog]: This master's current role is: LEADER
I20260808 08:25:37.800783 5654 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:25:37.801415 5654 catalog_manager.cc:1259] Loaded cluster ID: f69fef0bdad04b23b6271a7b0831ddac
I20260808 08:25:37.801479 5654 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:25:37.803840 5654 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:25:37.804494 5654 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3eaaf873a9b74e3d83276e19b8d7c150: Loaded TSK: 0
I20260808 08:25:37.805727 5654 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 08:25:37.884362 5644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:37.884647 5644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:37.884714 5644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:37.888693 5644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:37.888748 5644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:37.888834 5644 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.193
I20260808 08:25:37.893904 5644 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:46085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.193.keytab
--principal=oryx/127.22.235.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.193:45529
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.235.193
--webserver_port=41229
--tserver_master_addrs=127.22.235.254:42715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.193
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:37.894886 5644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:37.895834 5644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:37.906564 5692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:37.906553 5690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:37.906821 5644 server_base.cc:1034] running on GCE node
W20260808 08:25:37.906554 5688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:37.907407 5644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:37.908231 5644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:37.909439 5644 hybrid_clock.cc:648] HybridClock initialized: now 1786177537909405 us; error 40 us; skew 500 ppm
Aug 08 08:25:37 dist-test-slave-f7th krb5kdc[4640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177537, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.235.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:37.912856 5644 init.cc:377] Logged in from keytab as oryx/127.22.235.193@KRBTEST.COM (short username oryx)
I20260808 08:25:37.914014 5644 webserver.cc:460] Webserver started at http://127.22.235.193:41229/ using document root <none> and password file <none>
I20260808 08:25:37.914777 5644 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:37.914897 5644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:37.919863 5644 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 08:25:37.922354 5701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:37.923786 5644 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 08:25:37.923910 5644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-0/wal
uuid: "2340af5975234199b584739caef0460e"
format_stamp: "Formatted at 2026-08-08 08:25:36 on dist-test-slave-f7th"
server_key: "7ad616f248a786b94ba7a6961ffcaf60"
server_key_iv: "60d0771e33e00b168814a614af7009f6"
server_key_version: "encryptionkey@0"
I20260808 08:25:37.924584 5644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:37.952693 5644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:37.955528 5644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:37.955682 5644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:37.956228 5644 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:25:37.957098 5644 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:25:37.957157 5644 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:37.957201 5644 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:25:37.957221 5644 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:37.965890 5644 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.193:45529
I20260808 08:25:37.965909 5888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.193:45529 every 8 connection(s)
I20260808 08:25:37.966869 5644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
I20260808 08:25:37.975775 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 5644
I20260808 08:25:37.976275 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.194:37991
--local_ip_for_outbound_sockets=127.22.235.194
--tserver_master_addrs=127.22.235.254:42715
--webserver_port=36965
--webserver_interface=127.22.235.194
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab
--principal=kudu/127.22.235.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:46085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.22.235.194
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 08:25:37 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.193@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:37 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.193@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
W20260808 08:25:37.979552 5893 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:37.968887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:51709 (local address 127.22.235.254:42715)
0808 08:25:37.969402 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:37.969422 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:37.969600 (+ 178us) server_negotiation.cc:407] Connection header received
0808 08:25:37.971155 (+ 1555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:37.971166 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:37.971458 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:37.971756 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:37.972892 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:37.973836 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:37.974950 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:37.975407 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:37.975556 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:37.978865 (+ 3309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.193:51709: BlockingRecv error: recv got EOF from 127.22.235.193:51709 (error 108)
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":123,"threads_started":1}
I20260808 08:25:37.979564 5895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:37.968978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.193:51709)
0808 08:25:37.969446 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:37.969496 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:37.970744 (+ 1248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:37.971843 (+ 1099us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:37.971852 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:37.972259 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:37.972728 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:37.972743 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:37.973966 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:37.973969 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:37.974823 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:37.974833 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:37.975555 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:37.975559 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:37.975730 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:37.976367 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:37.976390 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:37.978637 (+ 2247us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":127,"threads_started":1}
W20260808 08:25:37.980207 5889 heartbeater.cc:644] Failed to heartbeat to 127.22.235.254:42715 (0 consecutive failures): Not authorized: Failed to ping master at 127.22.235.254:42715: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Aug 08 08:25:37 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.193@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:37 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.193@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:37.985531 5895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:37.980672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.193:58865)
0808 08:25:37.980798 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:37.980814 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:37.980895 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:37.981122 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:37.981125 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:37.981283 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:37.981394 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:37.981401 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:37.982064 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:37.982067 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:37.982869 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:37.982881 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:37.983261 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:37.983264 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:37.983355 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:37.983762 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:37.983776 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:37.985416 (+ 1640us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 08:25:37.985652 5893 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:37.980740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:58865 (local address 127.22.235.254:42715)
0808 08:25:37.980837 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:37.980839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:37.980855 (+ 16us) server_negotiation.cc:407] Connection header received
0808 08:25:37.980987 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:37.980989 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:37.981030 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:37.981086 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:37.981500 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:37.981965 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:37.982973 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:37.983171 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:37.983233 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:37.985557 (+ 2324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.193:58865: BlockingRecv error: recv got EOF from 127.22.235.193:58865 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 08 08:25:37 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.193@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:37 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.193@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:37.990465 5895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:37.986155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.193:57547)
0808 08:25:37.986282 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:37.986295 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:37.986361 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:37.986542 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:37.986544 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:37.986675 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:37.986740 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:37.986744 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:37.987406 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:37.987410 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:37.987899 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:37.987906 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:37.988320 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:37.988323 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:37.988392 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:37.988807 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:37.988818 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:37.990347 (+ 1529us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":78}
W20260808 08:25:37.990547 5893 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:37.986251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:57547 (local address 127.22.235.254:42715)
0808 08:25:37.986365 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:37.986369 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:37.986381 (+ 12us) server_negotiation.cc:407] Connection header received
0808 08:25:37.986417 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:37.986420 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:37.986457 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:37.986530 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:37.986853 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:37.987315 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:37.987998 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:37.988224 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:37.988318 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:37.990455 (+ 2137us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.193:57547: BlockingRecv error: recv got EOF from 127.22.235.193:57547 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260808 08:25:37.990835 5889 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 08:25:38.091894 5898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:38.092144 5898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:38.092192 5898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:38.097605 5898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:38.097680 5898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:38.097759 5898 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.194
I20260808 08:25:38.104161 5898 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:46085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.194.keytab
--principal=oryx/127.22.235.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.194:37991
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.235.194
--webserver_port=36965
--tserver_master_addrs=127.22.235.254:42715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5898
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.194
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:38.105363 5898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:38.106623 5898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:38.118803 5906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:38.118803 5905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:38.118815 5908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:38.119325 5898 server_base.cc:1034] running on GCE node
I20260808 08:25:38.119817 5898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:38.120533 5898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:38.121713 5898 hybrid_clock.cc:648] HybridClock initialized: now 1786177538121677 us; error 63 us; skew 500 ppm
Aug 08 08:25:38 dist-test-slave-f7th krb5kdc[4640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177538, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.235.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:38.125012 5898 init.cc:377] Logged in from keytab as oryx/127.22.235.194@KRBTEST.COM (short username oryx)
I20260808 08:25:38.126231 5898 webserver.cc:460] Webserver started at http://127.22.235.194:36965/ using document root <none> and password file <none>
I20260808 08:25:38.127028 5898 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:38.127146 5898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:38.132863 5898 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 08:25:38.135563 5924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:38.136922 5898 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 08:25:38.137032 5898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-1/wal
uuid: "3968a62d7d4141c9a80e559af53d5e29"
format_stamp: "Formatted at 2026-08-08 08:25:37 on dist-test-slave-f7th"
server_key: "795074592c7b659e0b038dbcee502e37"
server_key_iv: "7895694babce2fe9489d4e801e194469"
server_key_version: "encryptionkey@0"
I20260808 08:25:38.137579 5898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:38.150779 5898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:38.153638 5898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:38.153817 5898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:38.154441 5898 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:25:38.155292 5898 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:25:38.155339 5898 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:38.155395 5898 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:25:38.155423 5898 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:38.164130 5898 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.194:37991
I20260808 08:25:38.164187 6127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.194:37991 every 8 connection(s)
I20260808 08:25:38.165196 5898 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
I20260808 08:25:38.174258 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 5898
Aug 08 08:25:38 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.194@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:38.174726 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.195:45013
--local_ip_for_outbound_sockets=127.22.235.195
--tserver_master_addrs=127.22.235.254:42715
--webserver_port=36991
--webserver_interface=127.22.235.195
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab
--principal=kudu/127.22.235.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:46085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.22.235.195
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 08:25:38 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.194@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
W20260808 08:25:38.176117 5893 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:38.167029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:40983 (local address 127.22.235.254:42715)
0808 08:25:38.167187 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:38.167191 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:38.168007 (+ 816us) server_negotiation.cc:407] Connection header received
0808 08:25:38.169328 (+ 1321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:38.169332 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:38.169383 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:38.169456 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:38.170626 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:38.171223 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:38.172335 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:38.172554 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:38.172705 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:38.175978 (+ 3273us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.194:40983: BlockingRecv error: recv got EOF from 127.22.235.194:40983 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 08:25:38.176860 6136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:38.167414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.194:40983)
0808 08:25:38.167853 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:38.167893 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:38.169139 (+ 1246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:38.169624 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:38.169630 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:38.169975 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:38.170451 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:38.170465 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:38.171333 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:38.171336 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:38.172210 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:38.172220 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:38.172717 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:38.172720 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:38.172866 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:38.173428 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:38.173445 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:38.175827 (+ 2382us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":121,"threads_started":1}
W20260808 08:25:38.177359 6128 heartbeater.cc:644] Failed to heartbeat to 127.22.235.254:42715 (0 consecutive failures): Not authorized: Failed to ping master at 127.22.235.254:42715: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Aug 08 08:25:38 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.194@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:38 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.194@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:38.182957 6136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:38.177775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.194:49723)
0808 08:25:38.177899 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:38.177911 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:38.177990 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:38.178236 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:38.178238 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:38.178367 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:38.178459 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:38.178473 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:38.179402 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:38.179405 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:38.180002 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:38.180012 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:38.180551 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:38.180553 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:38.180655 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:38.181162 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:38.181174 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:38.182859 (+ 1685us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 08:25:38.183151 5893 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:38.177894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:49723 (local address 127.22.235.254:42715)
0808 08:25:38.178023 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:38.178027 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:38.178041 (+ 14us) server_negotiation.cc:407] Connection header received
0808 08:25:38.178082 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:38.178084 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:38.178126 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:38.178199 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:38.178574 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:38.179272 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:38.180124 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:38.180445 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:38.180553 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:38.183032 (+ 2479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.194:49723: BlockingRecv error: recv got EOF from 127.22.235.194:49723 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 08 08:25:38 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.194@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:38 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.194@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:38.188956 6136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:38.183661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.194:45493)
0808 08:25:38.183828 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:38.183842 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:38.183918 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:38.184253 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:38.184257 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:38.184421 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:38.184511 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:38.184517 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:38.185561 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:38.185563 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:38.186073 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:38.186081 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:38.186593 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:38.186595 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:38.186651 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:38.187135 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:38.187146 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:38.188830 (+ 1684us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104}
W20260808 08:25:38.189050 5893 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:38.183768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:45493 (local address 127.22.235.254:42715)
0808 08:25:38.183870 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:38.183873 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:38.183884 (+ 11us) server_negotiation.cc:407] Connection header received
0808 08:25:38.184007 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:38.184010 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:38.184056 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:38.184131 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:38.184692 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:38.185456 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:38.186181 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:38.186495 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:38.186604 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:38.188944 (+ 2340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.194:45493: BlockingRecv error: recv got EOF from 127.22.235.194:45493 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260808 08:25:38.189313 6128 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 08:25:38.290764 6138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:38.291024 6138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:38.291085 6138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:38.295030 6138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:38.295086 6138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:38.295162 6138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.195
I20260808 08:25:38.300495 6138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:46085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/krb5kdc/oryx_127.22.235.195.keytab
--principal=oryx/127.22.235.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.195:45013
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.235.195
--webserver_port=36991
--tserver_master_addrs=127.22.235.254:42715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.195
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:38.301502 6138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:38.302574 6138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:38.311733 6154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:38.311712 6152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:38.311744 6150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:38.311993 6138 server_base.cc:1034] running on GCE node
I20260808 08:25:38.312356 6138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:38.312980 6138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:38.314145 6138 hybrid_clock.cc:648] HybridClock initialized: now 1786177538314127 us; error 27 us; skew 500 ppm
Aug 08 08:25:38 dist-test-slave-f7th krb5kdc[4640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177538, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.235.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:38.319166 6138 init.cc:377] Logged in from keytab as oryx/127.22.235.195@KRBTEST.COM (short username oryx)
I20260808 08:25:38.320044 6138 webserver.cc:460] Webserver started at http://127.22.235.195:36991/ using document root <none> and password file <none>
I20260808 08:25:38.320566 6138 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:38.320673 6138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:38.324384 6138 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:25:38.326449 6166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:38.327617 6138 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 08:25:38.327716 6138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-2/wal
uuid: "fbc3be67bdcb41099cd16c94f40c7e58"
format_stamp: "Formatted at 2026-08-08 08:25:37 on dist-test-slave-f7th"
server_key: "b7199656bd3364832b6763c7fed16a69"
server_key_iv: "b3152ec58833ff057db65c9d76881f52"
server_key_version: "encryptionkey@0"
I20260808 08:25:38.328225 6138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:38.355022 6138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:38.358242 6138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:38.358440 6138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:38.359112 6138 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:25:38.360002 6138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:25:38.360051 6138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:38.360096 6138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:25:38.360126 6138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:38.368662 6138 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.195:45013
I20260808 08:25:38.368716 6367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.195:45013 every 8 connection(s)
I20260808 08:25:38.369649 6138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
I20260808 08:25:38.372882 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 6138
Aug 08 08:25:38 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.195@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:38 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.195@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
W20260808 08:25:38.381201 5893 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:38.372352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:46247 (local address 127.22.235.254:42715)
0808 08:25:38.372480 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:38.372484 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:38.372530 (+ 46us) server_negotiation.cc:407] Connection header received
0808 08:25:38.374047 (+ 1517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:38.374051 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:38.374104 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:38.374193 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:38.375540 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:38.376087 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:38.377549 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:38.377794 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:38.377869 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:38.381067 (+ 3198us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.195:46247: BlockingRecv error: recv got EOF from 127.22.235.195:46247 (error 108)
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":42,"spinlock_wait_cycles":1920}
I20260808 08:25:38.381938 6373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:38.371857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.195:46247)
0808 08:25:38.372381 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:38.372436 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:38.373837 (+ 1401us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:38.374362 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:38.374372 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:38.374838 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:38.375403 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:38.375417 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:38.376202 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:38.376206 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:38.377430 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:38.377441 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:38.377946 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:38.377951 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:38.378165 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:38.378678 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:38.378694 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:38.381002 (+ 2308us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":144,"threads_started":1}
I20260808 08:25:38.382318 6376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:38.373834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48862 (local address 127.22.235.254:42715)
0808 08:25:38.374067 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:38.374071 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:38.374081 (+ 10us) server_negotiation.cc:407] Connection header received
0808 08:25:38.374104 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:38.374106 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:38.374136 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:38.374212 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:38.375118 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:38.375708 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:38.376757 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:38.376976 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:38.377056 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:38.378600 (+ 1544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:38.378609 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:38.378627 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:38.378657 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:38.380213 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:38.380706 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:38.380711 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:38.380715 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:38.380834 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:38.381129 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:38.381132 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:38.381134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:38.381537 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:38.381678 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:38.381954 (+ 276us) server_negotiation.cc:299] Negotiation successful
0808 08:25:38.382197 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":170,"spinlock_wait_cycles":3840,"thread_start_us":84,"threads_started":1}
W20260808 08:25:38.382616 6370 heartbeater.cc:644] Failed to heartbeat to 127.22.235.254:42715 (0 consecutive failures): Not authorized: Failed to ping master at 127.22.235.254:42715: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Aug 08 08:25:38 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.195@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:38 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.195@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:38.387609 6373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:38.383046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.195:43117)
0808 08:25:38.383187 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:38.383201 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:38.383276 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:38.383585 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:38.383588 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:38.383717 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:38.383799 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:38.383804 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:38.384607 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:38.384639 (+ 32us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:38.385166 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:38.385175 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:38.385603 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:38.385607 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:38.385691 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:38.386009 (+ 318us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:38.386019 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:38.387494 (+ 1475us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":77,"mutex_wait_us":45}
W20260808 08:25:38.387719 6376 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:38.383100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:43117 (local address 127.22.235.254:42715)
0808 08:25:38.383404 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:38.383407 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:38.383417 (+ 10us) server_negotiation.cc:407] Connection header received
0808 08:25:38.383444 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:38.383446 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:38.383485 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:38.383546 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:38.383904 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:38.384511 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:38.385278 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:38.385509 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:38.385592 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:38.387628 (+ 2036us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.195:43117: BlockingRecv error: recv got EOF from 127.22.235.195:43117 (error 108)
Metrics: {"server-negotiator.queue_time_us":244}
Aug 08 08:25:38 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.195@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:38 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.195@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:38.391987 6373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:38.388104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.195:44481)
0808 08:25:38.388191 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:38.388204 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:38.388263 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:38.388441 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:38.388443 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:38.388563 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:38.388671 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:38.388677 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:38.389276 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:38.389278 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:38.389769 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:38.389776 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:38.390095 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:38.390097 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:38.390146 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:38.390513 (+ 367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:38.390524 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:38.391909 (+ 1385us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":26}
W20260808 08:25:38.392113 6376 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:38.388168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:44481 (local address 127.22.235.254:42715)
0808 08:25:38.388291 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:38.388294 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:38.388305 (+ 11us) server_negotiation.cc:407] Connection header received
0808 08:25:38.388332 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:38.388334 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:38.388365 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:38.388424 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:38.388752 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:38.389197 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:38.389842 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:38.390028 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:38.390089 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:38.392017 (+ 1928us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.195:44481: BlockingRecv error: recv got EOF from 127.22.235.195:44481 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260808 08:25:38.392182 6370 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 08 08:25:38 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.193@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:38 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.193@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:38.996194 6378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:38.991369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.193:45753)
0808 08:25:38.991633 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:38.991645 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:38.991737 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:38.991954 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:38.991956 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:38.992135 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:38.992233 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:38.992238 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:38.992935 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:38.992938 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:38.993515 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:38.993523 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:38.993879 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:38.993881 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:38.993937 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:38.994408 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:38.994420 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:38.996071 (+ 1651us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":77,"threads_started":1}
W20260808 08:25:38.996313 6382 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:38.991404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:45753 (local address 127.22.235.254:42715)
0808 08:25:38.991660 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:38.991663 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:38.991693 (+ 30us) server_negotiation.cc:407] Connection header received
0808 08:25:38.991811 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:38.991813 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:38.991856 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:38.991932 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:38.992315 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:38.992857 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:38.993598 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:38.993806 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:38.993870 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:38.996200 (+ 2330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.193:45753: BlockingRecv error: recv got EOF from 127.22.235.193:45753 (error 108)
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":82,"threads_started":1}
Aug 08 08:25:39 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.194@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:39 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.194@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:39.195057 6386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:39.189881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.194:55429)
0808 08:25:39.190132 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:39.190146 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:39.190247 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:39.190483 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:39.190487 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:39.190693 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:39.190803 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:39.190811 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:39.191650 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:39.191654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:39.192231 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:39.192241 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:39.192650 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:39.192654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:39.192734 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:39.193187 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:39.193198 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:39.194921 (+ 1723us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":71,"threads_started":1}
W20260808 08:25:39.195192 6382 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:39.189962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:55429 (local address 127.22.235.254:42715)
0808 08:25:39.190139 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:39.190143 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:39.190199 (+ 56us) server_negotiation.cc:407] Connection header received
0808 08:25:39.190320 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:39.190322 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:39.190366 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:39.190431 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:39.190926 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:39.191539 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:39.192326 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:39.192540 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:39.192635 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:39.195071 (+ 2436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.194:55429: BlockingRecv error: recv got EOF from 127.22.235.194:55429 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 08 08:25:39 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.195@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:39 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.195@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:39.398939 6387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:39.392828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.195:39329)
0808 08:25:39.393082 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:39.393096 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:39.393210 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:39.393483 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:39.393486 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:39.393738 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:39.393883 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:39.393893 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:39.394954 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:39.394957 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:39.395523 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:39.395531 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:39.396116 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:39.396120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:39.396213 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:39.396759 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:39.396774 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:39.398816 (+ 2042us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":83,"threads_started":1}
W20260808 08:25:39.399307 6382 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:39.392870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:39329 (local address 127.22.235.254:42715)
0808 08:25:39.393012 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:39.393017 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:39.393187 (+ 170us) server_negotiation.cc:407] Connection header received
0808 08:25:39.393310 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:39.393313 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:39.393374 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:39.393449 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:39.394015 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:39.394853 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:39.395618 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:39.396010 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:39.396119 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:39.399003 (+ 2884us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.195:39329: BlockingRecv error: recv got EOF from 127.22.235.195:39329 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 08 08:25:39 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.193@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:39 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.193@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:40.002458 6388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:39.996930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.193:34933)
0808 08:25:39.997226 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:39.997239 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:39.997329 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:39.997592 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:39.997594 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:39.997789 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:39.997886 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:39.997891 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:39.998639 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:39.998643 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:39.999325 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:39.999337 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:39.999789 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:39.999792 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:39.999857 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:40.000374 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:40.000386 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:40.002270 (+ 1884us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":108,"threads_started":1}
W20260808 08:25:40.002568 6390 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:39.996995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:34933 (local address 127.22.235.254:42715)
0808 08:25:39.997288 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:39.997292 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:39.997305 (+ 13us) server_negotiation.cc:407] Connection header received
0808 08:25:39.997428 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:39.997431 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:39.997491 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:39.997566 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:39.997974 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:39.998514 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:39.999446 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:39.999692 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:39.999765 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:40.002427 (+ 2662us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.193:34933: BlockingRecv error: recv got EOF from 127.22.235.193:34933 (error 108)
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":94,"threads_started":1}
Aug 08 08:25:40 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.194@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:40 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.194@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:40.201502 6391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:40.195808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.194:34533)
0808 08:25:40.196126 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:40.196139 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:40.196251 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:40.196497 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:40.196500 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:40.196784 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:40.196890 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:40.196897 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:40.197664 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:40.197667 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:40.198341 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:40.198350 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:40.198784 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:40.198787 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:40.198864 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:40.199335 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:40.199346 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:40.201357 (+ 2011us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":109,"threads_started":1}
W20260808 08:25:40.201598 6390 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:40.195911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:34533 (local address 127.22.235.254:42715)
0808 08:25:40.196082 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:40.196087 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:40.196209 (+ 122us) server_negotiation.cc:407] Connection header received
0808 08:25:40.196339 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:40.196342 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:40.196391 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:40.196460 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:40.197000 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:40.197556 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:40.198448 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:40.198682 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:40.198770 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:40.201490 (+ 2720us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.194:34533: BlockingRecv error: recv got EOF from 127.22.235.194:34533 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 08 08:25:40 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.195@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:40 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.195@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:40.405083 6394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:40.399648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.195:38049)
0808 08:25:40.399929 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:40.399944 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:40.400057 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:40.400299 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:40.400302 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:40.400527 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:40.400657 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:40.400664 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:40.401398 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:40.401402 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:40.401966 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:40.401975 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:40.402393 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:40.402397 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:40.402489 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:40.403036 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:40.403048 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:40.404927 (+ 1879us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":104,"threads_started":1}
W20260808 08:25:40.405148 6390 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:40.399784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:38049 (local address 127.22.235.254:42715)
0808 08:25:40.399923 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:40.399928 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:40.400003 (+ 75us) server_negotiation.cc:407] Connection header received
0808 08:25:40.400146 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:40.400149 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:40.400196 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:40.400260 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:40.400769 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:40.401292 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:40.402078 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:40.402292 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:40.402377 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:40.405055 (+ 2678us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.195:38049: BlockingRecv error: recv got EOF from 127.22.235.195:38049 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 08 08:25:41 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.193@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:41 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.193@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:41.008638 6395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:41.003252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.193:49325)
0808 08:25:41.003535 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:41.003548 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:41.003642 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:41.003895 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:41.003898 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:41.004093 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:41.004207 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:41.004216 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:41.005109 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:41.005113 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:41.005709 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:41.005719 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:41.006118 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:41.006121 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:41.006178 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:41.006637 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:41.006648 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:41.008436 (+ 1788us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":78,"threads_started":1}
W20260808 08:25:41.008726 6398 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:41.003346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:49325 (local address 127.22.235.254:42715)
0808 08:25:41.003628 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:41.003632 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:41.003645 (+ 13us) server_negotiation.cc:407] Connection header received
0808 08:25:41.003710 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:41.003712 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:41.003771 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:41.003844 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:41.004318 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:41.005012 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:41.005809 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:41.006033 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:41.006111 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:41.008581 (+ 2470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.193:49325: BlockingRecv error: recv got EOF from 127.22.235.193:49325 (error 108)
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":103,"threads_started":1}
Aug 08 08:25:41 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.194@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:41 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.194@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:41.207674 6399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:41.202206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.194:38149)
0808 08:25:41.202534 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:41.202547 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:41.202664 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:41.202988 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:41.202991 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:41.203194 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:41.203315 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:41.203322 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:41.204070 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:41.204074 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:41.204661 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:41.204670 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:41.205107 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:41.205111 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:41.205191 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:41.205670 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:41.205681 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:41.207494 (+ 1813us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":116,"threads_started":1}
W20260808 08:25:41.207738 6398 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:41.202310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:38149 (local address 127.22.235.254:42715)
0808 08:25:41.202485 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:41.202490 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:41.202623 (+ 133us) server_negotiation.cc:407] Connection header received
0808 08:25:41.202764 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:41.202771 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:41.202868 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:41.202939 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:41.203433 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:41.203967 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:41.204777 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:41.205004 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:41.205092 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:41.207638 (+ 2546us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.194:38149: BlockingRecv error: recv got EOF from 127.22.235.194:38149 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 08 08:25:41 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.195@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:41 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.195@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:41.410930 6400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:41.405775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.195:33193)
0808 08:25:41.406044 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:41.406057 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:41.406164 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:41.406396 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:41.406399 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:41.406602 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:41.406697 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:41.406703 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:41.407443 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:41.407448 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:41.408024 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:41.408033 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:41.408450 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:41.408454 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:41.408542 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:41.409049 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:41.409061 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:41.410814 (+ 1753us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":85,"threads_started":1}
W20260808 08:25:41.411140 6398 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:41.405854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:33193 (local address 127.22.235.254:42715)
0808 08:25:41.406001 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:41.406006 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:41.406117 (+ 111us) server_negotiation.cc:407] Connection header received
0808 08:25:41.406246 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:41.406249 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:41.406298 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:41.406363 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:41.406814 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:41.407340 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:41.408132 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:41.408352 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:41.408438 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:41.410940 (+ 2502us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.195:33193: BlockingRecv error: recv got EOF from 127.22.235.195:33193 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 08 08:25:42 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.193@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:42 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.193@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:42.014246 6404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:42.009364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.193:47903)
0808 08:25:42.009663 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:42.009676 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:42.009767 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:42.009991 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:42.009993 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:42.010176 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:42.010273 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:42.010278 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:42.010956 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:42.010958 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:42.011518 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:42.011525 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:42.011888 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:42.011890 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:42.011961 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:42.012430 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:42.012441 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:42.014127 (+ 1686us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":104,"threads_started":1}
W20260808 08:25:42.014387 6405 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:42.009443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:47903 (local address 127.22.235.254:42715)
0808 08:25:42.009706 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:42.009710 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:42.009723 (+ 13us) server_negotiation.cc:407] Connection header received
0808 08:25:42.009846 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:42.009848 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:42.009891 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:42.009977 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:42.010357 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:42.010881 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:42.011598 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:42.011809 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:42.011880 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:42.014264 (+ 2384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.193:47903: BlockingRecv error: recv got EOF from 127.22.235.193:47903 (error 108)
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":84,"threads_started":1}
Aug 08 08:25:42 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.194@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:42 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.194@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:42.213814 6406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:42.208575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.194:54225)
0808 08:25:42.208870 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:42.208883 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:42.208999 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:42.209251 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:42.209254 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:42.209503 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:42.209605 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:42.209611 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:42.210359 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:42.210362 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:42.210926 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:42.210935 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:42.211358 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:42.211362 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:42.211442 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:42.211905 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:42.211916 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:42.213663 (+ 1747us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":78,"threads_started":1}
W20260808 08:25:42.213907 6405 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:42.208667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:54225 (local address 127.22.235.254:42715)
0808 08:25:42.208829 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:42.208834 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:42.208955 (+ 121us) server_negotiation.cc:407] Connection header received
0808 08:25:42.209093 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:42.209096 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:42.209144 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:42.209214 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:42.209728 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:42.210252 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:42.211042 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:42.211258 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:42.211345 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:42.213805 (+ 2460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.194:54225: BlockingRecv error: recv got EOF from 127.22.235.194:54225 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 08 08:25:42 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.195@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:42 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.195@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:42.417164 6407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:42.411669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.195:41533)
0808 08:25:42.412017 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:42.412029 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:42.412135 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:42.412401 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:42.412404 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:42.412700 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:42.412828 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:42.412836 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:42.413720 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:42.413723 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:42.414266 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:42.414274 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:42.414638 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:42.414640 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:42.414696 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:42.415189 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:42.415203 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:42.417015 (+ 1812us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":94,"threads_started":1}
W20260808 08:25:42.417359 6405 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:42.411738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:41533 (local address 127.22.235.254:42715)
0808 08:25:42.411900 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:42.411905 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:42.412100 (+ 195us) server_negotiation.cc:407] Connection header received
0808 08:25:42.412223 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:42.412227 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:42.412274 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:42.412349 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:42.412942 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:42.413621 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:42.414348 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:42.414563 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:42.414629 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:42.417155 (+ 2526us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.195:41533: BlockingRecv error: recv got EOF from 127.22.235.195:41533 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 08 08:25:43 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.193@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:43 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.193@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:43.020998 6410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:43.015131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.193:38643)
0808 08:25:43.015479 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:43.015507 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:43.015614 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:43.015879 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:43.015881 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:43.016059 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:43.016155 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:43.016160 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:43.017255 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:43.017258 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:43.017826 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:43.017835 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:43.018408 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:43.018411 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:43.018472 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:43.018971 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:43.018984 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:43.020880 (+ 1896us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":80,"threads_started":1}
W20260808 08:25:43.021296 6411 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:43.015238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:38643 (local address 127.22.235.254:42715)
0808 08:25:43.015482 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:43.015487 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:43.015582 (+ 95us) server_negotiation.cc:407] Connection header received
0808 08:25:43.015707 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:43.015711 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:43.015764 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:43.015860 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:43.016255 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:43.017143 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:43.017930 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:43.018300 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:43.018395 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:43.021056 (+ 2661us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.193:38643: BlockingRecv error: recv got EOF from 127.22.235.193:38643 (error 108)
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":69,"threads_started":1}
Aug 08 08:25:43 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.194@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:43 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.194@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:43.220366 6413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:43.214651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.194:51423)
0808 08:25:43.215219 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:43.215232 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:43.215336 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:43.215667 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:43.215670 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:43.215882 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:43.215977 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:43.215984 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:43.216763 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:43.216767 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:43.217355 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:43.217366 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:43.217794 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:43.217798 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:43.217884 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:43.218359 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:43.218371 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:43.220243 (+ 1872us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":410,"threads_started":1}
W20260808 08:25:43.220482 6411 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:43.214782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:51423 (local address 127.22.235.254:42715)
0808 08:25:43.214906 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:43.214909 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:43.215305 (+ 396us) server_negotiation.cc:407] Connection header received
0808 08:25:43.215439 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:43.215443 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:43.215548 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:43.215626 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:43.216100 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:43.216666 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:43.217457 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:43.217685 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:43.217770 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:43.220374 (+ 2604us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.194:51423: BlockingRecv error: recv got EOF from 127.22.235.194:51423 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 08 08:25:43 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.195@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:43 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.195@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:43.425982 6415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:43.417862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.195:39141)
0808 08:25:43.418167 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:43.418180 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:43.418287 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:43.418542 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:43.418545 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:43.418749 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:43.418839 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:43.418845 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:43.419642 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:43.419646 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:43.420225 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:43.420234 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:43.420698 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:43.420702 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:43.420785 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:43.423934 (+ 3149us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:43.423946 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:43.425796 (+ 1850us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":91,"threads_started":1}
W20260808 08:25:43.426044 6411 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:43.417963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:39141 (local address 127.22.235.254:42715)
0808 08:25:43.418105 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:43.418110 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:43.418248 (+ 138us) server_negotiation.cc:407] Connection header received
0808 08:25:43.418374 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:43.418378 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:43.418424 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:43.418488 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:43.418957 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:43.419536 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:43.420343 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:43.420563 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:43.420681 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:43.425941 (+ 5260us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.195:39141: BlockingRecv error: recv got EOF from 127.22.235.195:39141 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 08 08:25:44 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.193@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:44 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.193@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:44.027024 6416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:44.021707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.193:53571)
0808 08:25:44.022032 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:44.022047 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:44.022161 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:44.022415 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:44.022418 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:44.022635 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:44.022731 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:44.022736 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:44.023498 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:44.023502 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:44.024056 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:44.024064 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:44.024503 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:44.024507 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:44.024589 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:44.025129 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:44.025141 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:44.026873 (+ 1732us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":82,"threads_started":1}
W20260808 08:25:44.027140 6417 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:44.021758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:53571 (local address 127.22.235.254:42715)
0808 08:25:44.022008 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:44.022013 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:44.022118 (+ 105us) server_negotiation.cc:407] Connection header received
0808 08:25:44.022250 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:44.022253 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:44.022303 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:44.022378 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:44.022850 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:44.023387 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:44.024169 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:44.024397 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:44.024482 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:44.027022 (+ 2540us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.193:53571: BlockingRecv error: recv got EOF from 127.22.235.193:53571 (error 108)
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":75,"threads_started":1}
Aug 08 08:25:44 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.194@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:44 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.194@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:44.226159 6420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:44.221105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.194:58477)
0808 08:25:44.221367 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:44.221380 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:44.221497 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:44.221721 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:44.221723 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:44.221909 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:44.222008 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:44.222014 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:44.222699 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:44.222702 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:44.223258 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:44.223265 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:44.223628 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:44.223630 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:44.223686 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:44.224135 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:44.224145 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:44.226043 (+ 1898us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":100,"threads_started":1}
W20260808 08:25:44.226301 6417 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:44.221173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:58477 (local address 127.22.235.254:42715)
0808 08:25:44.221310 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:44.221315 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:44.221444 (+ 129us) server_negotiation.cc:407] Connection header received
0808 08:25:44.221571 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:44.221573 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:44.221618 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:44.221681 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:44.222094 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:44.222618 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:44.223337 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:44.223550 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:44.223618 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:44.226175 (+ 2557us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.194:58477: BlockingRecv error: recv got EOF from 127.22.235.194:58477 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 08:25:44 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.195@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:44 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.195@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:44.432096 6421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:44.426805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.195:46533)
0808 08:25:44.427099 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:44.427111 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:44.427222 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:44.427468 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:44.427471 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:44.427681 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:44.427779 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:44.427785 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:44.428523 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:44.428526 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:44.429145 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:44.429157 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:44.429592 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:44.429596 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:44.429677 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:44.430170 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:44.430186 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:44.431968 (+ 1782us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":87,"threads_started":1}
W20260808 08:25:44.432276 6417 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:44.426900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:46533 (local address 127.22.235.254:42715)
0808 08:25:44.427038 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:44.427043 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:44.427179 (+ 136us) server_negotiation.cc:407] Connection header received
0808 08:25:44.427312 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:44.427315 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:44.427364 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:44.427432 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:44.427897 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:44.428418 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:44.429265 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:44.429492 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:44.429577 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:44.432145 (+ 2568us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.195:46533: BlockingRecv error: recv got EOF from 127.22.235.195:46533 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 08 08:25:45 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.193@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:45 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.193@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:45.033188 6422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:45.027798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.193:53431)
0808 08:25:45.028110 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:45.028124 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:45.028220 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:45.028442 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:45.028444 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:45.028685 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:45.028789 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:45.028794 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:45.029529 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:45.029532 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:45.030084 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:45.030093 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:45.030491 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:45.030496 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:45.030584 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:45.031105 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:45.031117 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:45.033038 (+ 1921us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":73,"threads_started":1}
W20260808 08:25:45.033273 6423 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:45.027908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:53431 (local address 127.22.235.254:42715)
0808 08:25:45.028144 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:45.028147 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:45.028174 (+ 27us) server_negotiation.cc:407] Connection header received
0808 08:25:45.028300 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:45.028302 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:45.028346 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:45.028413 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:45.028879 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:45.029429 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:45.030170 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:45.030382 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:45.030447 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:45.033157 (+ 2710us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.193:53431: BlockingRecv error: recv got EOF from 127.22.235.193:53431 (error 108)
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":76,"threads_started":1}
Aug 08 08:25:45 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.194@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:45 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.194@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:45.231900 6424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:45.226827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.194:38167)
0808 08:25:45.227144 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:45.227157 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:45.227257 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:45.227454 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:45.227456 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:45.227635 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:45.227730 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:45.227735 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:45.228408 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:45.228410 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:45.229008 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:45.229019 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:45.229391 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:45.229393 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:45.229449 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:45.229944 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:45.229956 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:45.231778 (+ 1822us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":81,"threads_started":1}
W20260808 08:25:45.232034 6423 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:45.226888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:38167 (local address 127.22.235.254:42715)
0808 08:25:45.227025 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:45.227029 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:45.227212 (+ 183us) server_negotiation.cc:407] Connection header received
0808 08:25:45.227319 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:45.227321 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:45.227364 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:45.227428 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:45.227809 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:45.228331 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:45.229104 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:45.229314 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:45.229381 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:45.231900 (+ 2519us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.194:38167: BlockingRecv error: recv got EOF from 127.22.235.194:38167 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 08 08:25:45 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.195@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:45 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.195@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:45.438100 6427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:45.432853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.195:33111)
0808 08:25:45.433175 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:45.433188 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:45.433309 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:45.433559 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:45.433562 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:45.433776 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:45.433874 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:45.433880 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:45.434664 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:45.434668 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:45.435268 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:45.435279 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:45.435681 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:45.435684 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:45.435742 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:45.436234 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:45.436247 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:45.437969 (+ 1722us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":84,"threads_started":1}
W20260808 08:25:45.438227 6423 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:45.432883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:33111 (local address 127.22.235.254:42715)
0808 08:25:45.433055 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:45.433059 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:45.433247 (+ 188us) server_negotiation.cc:407] Connection header received
0808 08:25:45.433396 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:45.433400 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:45.433453 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:45.433522 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:45.433997 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:45.434527 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:45.435365 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:45.435580 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:45.435652 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:45.438108 (+ 2456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.195:33111: BlockingRecv error: recv got EOF from 127.22.235.195:33111 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 08 08:25:46 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.193@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:46 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.193@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:46.038939 6428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:46.033901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.193:52101)
0808 08:25:46.034240 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:46.034255 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:46.034342 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:46.034562 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:46.034563 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:46.034743 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:46.034837 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:46.034842 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:46.035521 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:46.035523 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:46.036117 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:46.036128 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:46.036541 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:46.036543 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:46.036602 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:46.037076 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:46.037087 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:46.038823 (+ 1736us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":101,"threads_started":1}
W20260808 08:25:46.039105 6429 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:46.034041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:52101 (local address 127.22.235.254:42715)
0808 08:25:46.034299 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:46.034302 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:46.034315 (+ 13us) server_negotiation.cc:407] Connection header received
0808 08:25:46.034418 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:46.034420 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:46.034463 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:46.034534 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:46.034916 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:46.035442 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:46.036225 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:46.036448 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:46.036535 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:46.038960 (+ 2425us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.193:52101: BlockingRecv error: recv got EOF from 127.22.235.193:52101 (error 108)
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":93,"threads_started":1}
Aug 08 08:25:46 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.194@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:46 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.194@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:46.238093 6430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:46.232765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.194:41157)
0808 08:25:46.233015 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:46.233028 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:46.233127 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:46.233329 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:46.233331 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:46.233514 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:46.233612 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:46.233617 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:46.234418 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:46.234420 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:46.234976 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:46.234982 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:46.235347 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:46.235349 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:46.235407 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:46.235860 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:46.235871 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:46.237975 (+ 2104us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":71,"threads_started":1}
W20260808 08:25:46.238210 6429 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:46.232778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:41157 (local address 127.22.235.254:42715)
0808 08:25:46.232897 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:46.232901 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:46.233088 (+ 187us) server_negotiation.cc:407] Connection header received
0808 08:25:46.233195 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:46.233197 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:46.233241 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:46.233306 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:46.233692 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:46.234343 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:46.235063 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:46.235268 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:46.235334 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:46.238107 (+ 2773us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.194:41157: BlockingRecv error: recv got EOF from 127.22.235.194:41157 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 08 08:25:46 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.195@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:46 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.195@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:46.444267 6433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:46.438815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.195:34233)
0808 08:25:46.439073 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:46.439088 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:46.439196 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:46.439397 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:46.439399 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:46.439647 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:46.439793 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:46.439800 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:46.440694 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:46.440698 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:46.441395 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:46.441407 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:46.441916 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:46.441920 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:46.442007 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:46.442441 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:46.442452 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:46.444103 (+ 1651us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":73,"threads_started":1}
W20260808 08:25:46.444343 6429 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:46.438896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:34233 (local address 127.22.235.254:42715)
0808 08:25:46.439031 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:46.439035 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:46.439147 (+ 112us) server_negotiation.cc:407] Connection header received
0808 08:25:46.439260 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:46.439262 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:46.439307 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:46.439375 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:46.439915 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:46.440551 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:46.441517 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:46.441800 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:46.441893 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:46.444240 (+ 2347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.195:34233: BlockingRecv error: recv got EOF from 127.22.235.195:34233 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 08 08:25:47 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.193@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:47 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.193@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:47.045390 6435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:47.039701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.193:47845)
0808 08:25:47.039955 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:47.039984 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:47.040073 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:47.040303 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:47.040305 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:47.040506 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:47.040605 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:47.040646 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:47.041504 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:47.041507 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:47.042130 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:47.042141 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:47.042595 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:47.042599 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:47.042680 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:47.043207 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:47.043219 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:47.045270 (+ 2051us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":76,"threads_started":1}
W20260808 08:25:47.045558 6436 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:47.039824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:47845 (local address 127.22.235.254:42715)
0808 08:25:47.040043 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:47.040046 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:47.040060 (+ 14us) server_negotiation.cc:407] Connection header received
0808 08:25:47.040149 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:47.040151 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:47.040198 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:47.040276 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:47.040744 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:47.041397 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:47.042240 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:47.042495 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:47.042577 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:47.045428 (+ 2851us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.193:47845: BlockingRecv error: recv got EOF from 127.22.235.193:47845 (error 108)
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":67,"threads_started":1}
Aug 08 08:25:47 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.194@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:47 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.194@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:47.244139 6438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:47.238890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.194:33683)
0808 08:25:47.239146 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:47.239159 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:47.239257 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:47.239468 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:47.239471 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:47.239651 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:47.239748 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:47.239754 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:47.240519 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:47.240522 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:47.241108 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:47.241117 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:47.241625 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:47.241627 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:47.241691 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:47.242176 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:47.242188 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:47.244013 (+ 1825us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":82,"threads_started":1}
W20260808 08:25:47.244239 6436 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:47.238961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:33683 (local address 127.22.235.254:42715)
0808 08:25:47.239108 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:47.239111 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:47.239219 (+ 108us) server_negotiation.cc:407] Connection header received
0808 08:25:47.239340 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:47.239342 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:47.239386 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:47.239443 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:47.239837 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:47.240429 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:47.241247 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:47.241508 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:47.241586 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:47.244139 (+ 2553us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.194:33683: BlockingRecv error: recv got EOF from 127.22.235.194:33683 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 08 08:25:47 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.195@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:47 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.195@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:47.450160 6440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:47.445096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.195:50279)
0808 08:25:47.445373 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:47.445386 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:47.445480 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:47.445698 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:47.445699 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:47.445878 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:47.445978 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:47.445983 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:47.446703 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:47.446706 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:47.447274 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:47.447283 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:47.447662 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:47.447664 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:47.447722 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:47.448180 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:47.448190 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:47.449994 (+ 1804us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":109,"threads_started":1}
W20260808 08:25:47.450280 6436 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:47.445153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:50279 (local address 127.22.235.254:42715)
0808 08:25:47.445287 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:47.445292 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:47.445438 (+ 146us) server_negotiation.cc:407] Connection header received
0808 08:25:47.445560 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:47.445562 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:47.445609 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:47.445675 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:47.446082 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:47.446602 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:47.447359 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:47.447569 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:47.447636 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:47.450177 (+ 2541us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.195:50279: BlockingRecv error: recv got EOF from 127.22.235.195:50279 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 08 08:25:48 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.193@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:48 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.193@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:48.054666 6441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:48.046134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.193:49805)
0808 08:25:48.046409 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:48.046422 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:48.046539 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:48.046757 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:48.046759 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:48.046947 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:48.047045 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:48.047050 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:48.047731 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:48.047733 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:48.048284 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:48.048291 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:48.048685 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:48.048687 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:48.048747 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:48.052842 (+ 4095us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:48.052853 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:48.054543 (+ 1690us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":86,"threads_started":1}
W20260808 08:25:48.054821 6442 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:48.046218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:49805 (local address 127.22.235.254:42715)
0808 08:25:48.046438 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:48.046442 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:48.046487 (+ 45us) server_negotiation.cc:407] Connection header received
0808 08:25:48.046614 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:48.046616 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:48.046659 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:48.046732 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:48.047132 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:48.047653 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:48.048363 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:48.048572 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:48.048660 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:48.054684 (+ 6024us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.193:49805: BlockingRecv error: recv got EOF from 127.22.235.193:49805 (error 108)
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":57,"threads_started":1}
Aug 08 08:25:48 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.194@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:48 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.194@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:48.250120 6443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:48.244872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.194:39263)
0808 08:25:48.245158 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:48.245172 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:48.245267 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:48.245468 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:48.245470 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:48.245650 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:48.245768 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:48.245775 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:48.246491 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:48.246493 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:48.247068 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:48.247076 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:48.247439 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:48.247441 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:48.247502 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:48.247952 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:48.247973 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:48.249951 (+ 1978us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":84,"threads_started":1}
W20260808 08:25:48.250193 6442 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:48.244949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:39263 (local address 127.22.235.254:42715)
0808 08:25:48.245102 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:48.245106 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:48.245231 (+ 125us) server_negotiation.cc:407] Connection header received
0808 08:25:48.245343 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:48.245345 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:48.245386 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:48.245443 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:48.245870 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:48.246403 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:48.247151 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:48.247360 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:48.247429 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:48.250093 (+ 2664us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.194:39263: BlockingRecv error: recv got EOF from 127.22.235.194:39263 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 08 08:25:48 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.195@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:48 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.195@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:48.459404 6445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:48.451085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.195:46135)
0808 08:25:48.451342 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:48.451355 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:48.451451 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:48.451661 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:48.451663 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:48.451841 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:48.451937 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:48.451942 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:48.452644 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:48.452647 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:48.453265 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:48.453277 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:48.453834 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:48.453837 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:48.453924 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:48.457596 (+ 3672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:48.457609 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:48.459286 (+ 1677us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":86,"threads_started":1}
W20260808 08:25:48.459547 6442 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:48.451204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:46135 (local address 127.22.235.254:42715)
0808 08:25:48.451343 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:48.451347 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:48.451417 (+ 70us) server_negotiation.cc:407] Connection header received
0808 08:25:48.451532 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:48.451535 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:48.451581 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:48.451647 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:48.452019 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:48.452541 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:48.453388 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:48.453707 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:48.453807 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:48.459414 (+ 5607us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.195:46135: BlockingRecv error: recv got EOF from 127.22.235.195:46135 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 08 08:25:49 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.193@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:49 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.193@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:49.060976 6446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:49.055514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.193:45123)
0808 08:25:49.055781 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:49.055796 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:49.055908 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:49.056194 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:49.056197 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:49.056406 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:49.056542 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:49.056552 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:49.057408 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:49.057411 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:49.057970 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:49.057978 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:49.058379 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:49.058381 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:49.058437 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:49.058908 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:49.058919 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:49.060833 (+ 1914us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":76,"threads_started":1}
W20260808 08:25:49.061087 6447 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:49.055606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:45123 (local address 127.22.235.254:42715)
0808 08:25:49.055897 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:49.055902 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:49.055918 (+ 16us) server_negotiation.cc:407] Connection header received
0808 08:25:49.055999 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:49.056002 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:49.056059 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:49.056144 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:49.056662 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:49.057307 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:49.058073 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:49.058294 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:49.058373 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:49.060974 (+ 2601us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.193:45123: BlockingRecv error: recv got EOF from 127.22.235.193:45123 (error 108)
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":76,"threads_started":1}
Aug 08 08:25:49 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.194@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:49 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.194@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:49.257359 6448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:49.250904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.194:49313)
0808 08:25:49.251642 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:49.251658 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:49.251782 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:49.252214 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:49.252218 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:49.252448 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:49.252576 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:49.252585 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:49.253588 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:49.253592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:49.254262 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:49.254272 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:49.254792 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:49.254795 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:49.254878 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:49.255376 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:49.255392 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:49.257194 (+ 1802us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":78,"threads_started":1}
W20260808 08:25:49.257552 6447 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:49.250976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:49313 (local address 127.22.235.254:42715)
0808 08:25:49.251121 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:49.251127 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:49.251738 (+ 611us) server_negotiation.cc:407] Connection header received
0808 08:25:49.251894 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:49.251898 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:49.251961 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:49.252060 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:49.252719 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:49.253490 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:49.254374 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:49.254679 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:49.254768 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:49.257338 (+ 2570us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.194:49313: BlockingRecv error: recv got EOF from 127.22.235.194:49313 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 08 08:25:49 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.195@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:49 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.195@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:49.465691 6449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:49.460194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.195:51515)
0808 08:25:49.460526 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:49.460540 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:49.460663 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:49.460878 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:49.460880 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:49.461061 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:49.461158 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:49.461163 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:49.462020 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:49.462024 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:49.462745 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:49.462758 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:49.463232 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:49.463235 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:49.463298 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:49.463766 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:49.463776 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:49.465542 (+ 1766us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":109,"threads_started":1}
W20260808 08:25:49.465783 6447 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:49.460307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:51515 (local address 127.22.235.254:42715)
0808 08:25:49.460449 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:49.460453 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:49.460604 (+ 151us) server_negotiation.cc:407] Connection header received
0808 08:25:49.460731 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:49.460734 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:49.460782 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:49.460851 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:49.461241 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:49.461902 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:49.462857 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:49.463136 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:49.463217 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:49.465681 (+ 2464us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.195:51515: BlockingRecv error: recv got EOF from 127.22.235.195:51515 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 08 08:25:50 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.193@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:50 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.193@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:50.066663 6454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:50.061716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.193:47441)
0808 08:25:50.061989 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:50.062002 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:50.062104 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:50.062312 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:50.062314 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:50.062493 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:50.062589 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:50.062595 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:50.063261 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:50.063263 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:50.063807 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:50.063814 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:50.064206 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:50.064209 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:50.064265 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:50.064800 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:50.064813 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:50.066544 (+ 1731us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":76,"threads_started":1}
W20260808 08:25:50.066810 6455 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:50.061776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:47441 (local address 127.22.235.254:42715)
0808 08:25:50.062014 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:50.062018 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:50.062064 (+ 46us) server_negotiation.cc:407] Connection header received
0808 08:25:50.062173 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:50.062175 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:50.062216 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:50.062290 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:50.062672 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:50.063187 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:50.063904 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:50.064127 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:50.064199 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:50.066668 (+ 2469us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.193:47441: BlockingRecv error: recv got EOF from 127.22.235.193:47441 (error 108)
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":68,"threads_started":1}
Aug 08 08:25:50 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.194@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:50 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.194@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:50.263114 6457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:50.258107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.194:55707)
0808 08:25:50.258383 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:50.258398 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:50.258514 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:50.258714 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:50.258716 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:50.258893 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:50.258988 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:50.258992 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:50.259643 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:50.259646 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:50.260183 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:50.260189 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:50.260559 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:50.260561 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:50.260647 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:50.261145 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:50.261158 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:50.262963 (+ 1805us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":98,"threads_started":1}
W20260808 08:25:50.263217 6455 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:50.258186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:55707 (local address 127.22.235.254:42715)
0808 08:25:50.258320 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:50.258324 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:50.258461 (+ 137us) server_negotiation.cc:407] Connection header received
0808 08:25:50.258584 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:50.258586 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:50.258630 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:50.258701 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:50.259068 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:50.259567 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:50.260264 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:50.260475 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:50.260547 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:50.263098 (+ 2551us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.194:55707: BlockingRecv error: recv got EOF from 127.22.235.194:55707 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 08 08:25:50 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.195@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:50 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.195@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:50.471700 6458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:50.466519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.195:41493)
0808 08:25:50.466849 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:50.466862 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:50.466967 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:50.467261 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:50.467264 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:50.467467 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:50.467581 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:50.467588 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:50.468325 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:50.468330 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:50.468918 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:50.468927 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:50.469357 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:50.469361 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:50.469442 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:50.469881 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:50.469892 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:50.471584 (+ 1692us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":109,"threads_started":1}
W20260808 08:25:50.471833 6455 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:50.466620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:41493 (local address 127.22.235.254:42715)
0808 08:25:50.466751 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:50.466756 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:50.466927 (+ 171us) server_negotiation.cc:407] Connection header received
0808 08:25:50.467090 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:50.467093 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:50.467147 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:50.467214 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:50.467695 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:50.468223 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:50.469031 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:50.469255 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:50.469341 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:50.471722 (+ 2381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.195:41493: BlockingRecv error: recv got EOF from 127.22.235.195:41493 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 08 08:25:51 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.193@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:51 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.193@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:51.072717 6459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:51.067384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.193:50875)
0808 08:25:51.067665 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:51.067679 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:51.067777 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:51.068052 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:51.068055 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:51.068253 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:51.068357 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:51.068363 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:51.069106 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:51.069109 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:51.069673 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:51.069682 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:51.070116 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:51.070119 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:51.070186 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:51.070700 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:51.070712 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:51.072507 (+ 1795us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":102,"threads_started":1}
W20260808 08:25:51.072791 6460 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:51.067440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:50875 (local address 127.22.235.254:42715)
0808 08:25:51.067691 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:51.067695 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:51.067746 (+ 51us) server_negotiation.cc:407] Connection header received
0808 08:25:51.067849 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:51.067852 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:51.067896 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:51.067971 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:51.068453 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:51.069009 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:51.069782 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:51.070007 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:51.070082 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:51.072676 (+ 2594us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.193:50875: BlockingRecv error: recv got EOF from 127.22.235.193:50875 (error 108)
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":74,"threads_started":1}
Aug 08 08:25:51 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.194@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:51 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.194@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:51.268982 6461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:51.263843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.194:48219)
0808 08:25:51.264197 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:51.264210 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:51.264307 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:51.264517 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:51.264519 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:51.264740 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:51.264841 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:51.264847 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:51.265550 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:51.265552 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:51.266100 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:51.266107 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:51.266474 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:51.266477 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:51.266535 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:51.267044 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:51.267056 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:51.268822 (+ 1766us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":114,"threads_started":1}
W20260808 08:25:51.269070 6460 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:51.263977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:48219 (local address 127.22.235.254:42715)
0808 08:25:51.264126 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:51.264130 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:51.264270 (+ 140us) server_negotiation.cc:407] Connection header received
0808 08:25:51.264384 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:51.264386 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:51.264433 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:51.264490 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:51.264931 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:51.265467 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:51.266176 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:51.266378 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:51.266443 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:51.268958 (+ 2515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.194:48219: BlockingRecv error: recv got EOF from 127.22.235.194:48219 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 08 08:25:51 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.195@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:51 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.195@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:51.477809 6462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:51.472439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.195:59507)
0808 08:25:51.472753 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:51.472765 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:51.472866 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:51.473098 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:51.473100 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:51.473281 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:51.473377 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:51.473382 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:51.474084 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:51.474086 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:51.474760 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:51.474772 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:51.475327 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:51.475331 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:51.475410 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:51.475918 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:51.475930 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:51.477686 (+ 1756us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":102,"threads_started":1}
W20260808 08:25:51.477910 6460 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:51.472553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:59507 (local address 127.22.235.254:42715)
0808 08:25:51.472701 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:51.472704 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:51.472824 (+ 120us) server_negotiation.cc:407] Connection header received
0808 08:25:51.472948 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:51.472950 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:51.473010 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:51.473067 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:51.473485 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:51.474005 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:51.474870 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:51.475180 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:51.475273 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:51.477811 (+ 2538us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.195:59507: BlockingRecv error: recv got EOF from 127.22.235.195:59507 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 08 08:25:52 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.193@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:52 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.193@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:52.078907 6463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:52.073601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.193:43535)
0808 08:25:52.073877 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:52.073893 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:52.073993 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:52.074193 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:52.074195 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:52.074390 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:52.074488 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:52.074502 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:52.075216 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:52.075219 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:52.075906 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:52.075919 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:52.076448 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:52.076452 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:52.076527 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:52.077018 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:52.077030 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:52.078779 (+ 1749us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":78,"threads_started":1}
W20260808 08:25:52.079066 6464 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:52.073684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:43535 (local address 127.22.235.254:42715)
0808 08:25:52.073992 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:52.073996 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:52.074010 (+ 14us) server_negotiation.cc:407] Connection header received
0808 08:25:52.074051 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:52.074053 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:52.074094 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:52.074170 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:52.074585 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:52.075106 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:52.076035 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:52.076325 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:52.076431 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:52.078918 (+ 2487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.193:43535: BlockingRecv error: recv got EOF from 127.22.235.193:43535 (error 108)
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":79,"threads_started":1}
Aug 08 08:25:52 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.194@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:52 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.194@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:52.274674 6465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:52.269738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.194:50909)
0808 08:25:52.270000 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:52.270013 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:52.270117 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:52.270323 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:52.270324 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:52.270527 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:52.270620 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:52.270625 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:52.271296 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:52.271298 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:52.271856 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:52.271864 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:52.272257 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:52.272259 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:52.272316 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:52.272832 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:52.272844 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:52.274548 (+ 1704us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":76,"threads_started":1}
W20260808 08:25:52.274780 6464 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:52.269825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:50909 (local address 127.22.235.254:42715)
0808 08:25:52.269983 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:52.270000 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:52.270076 (+ 76us) server_negotiation.cc:407] Connection header received
0808 08:25:52.270186 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:52.270188 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:52.270232 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:52.270295 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:52.270701 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:52.271219 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:52.271936 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:52.272158 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:52.272228 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:52.274680 (+ 2452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.194:50909: BlockingRecv error: recv got EOF from 127.22.235.194:50909 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 08 08:25:52 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.195@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:52 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.195@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:52.484156 6466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:52.478562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.195:43589)
0808 08:25:52.478842 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:52.478854 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:52.478962 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:52.479207 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:52.479210 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:52.479417 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:52.479531 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:52.479538 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:52.480287 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:52.480291 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:52.480933 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:52.480945 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:52.481393 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:52.481397 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:52.481483 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:52.481979 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:52.481992 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:52.484000 (+ 2008us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":92,"threads_started":1}
W20260808 08:25:52.484232 6464 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:52.478646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:43589 (local address 127.22.235.254:42715)
0808 08:25:52.478780 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:52.478784 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:52.478920 (+ 136us) server_negotiation.cc:407] Connection header received
0808 08:25:52.479050 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:52.479053 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:52.479105 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:52.479169 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:52.479646 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:52.480183 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:52.481041 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:52.481270 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:52.481357 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:52.484134 (+ 2777us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.195:43589: BlockingRecv error: recv got EOF from 127.22.235.195:43589 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 08 08:25:53 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.193@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:53 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.193@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:53.085124 6468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:53.079779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.193:35929)
0808 08:25:53.080061 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:53.080075 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:53.080168 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:53.080380 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:53.080382 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:53.080576 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:53.080716 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:53.080723 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:53.081493 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:53.081496 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:53.082109 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:53.082124 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:53.082590 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:53.082594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:53.082679 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:53.083123 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:53.083135 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:53.085008 (+ 1873us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":77,"threads_started":1}
W20260808 08:25:53.085274 6469 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:53.079854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:35929 (local address 127.22.235.254:42715)
0808 08:25:53.080090 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:53.080094 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:53.080126 (+ 32us) server_negotiation.cc:407] Connection header received
0808 08:25:53.080239 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:53.080241 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:53.080282 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:53.080358 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:53.080815 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:53.081395 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:53.082239 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:53.082482 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:53.082573 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:53.085150 (+ 2577us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.193:35929: BlockingRecv error: recv got EOF from 127.22.235.193:35929 (error 108)
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":70,"threads_started":1}
Aug 08 08:25:53 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.194@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 08:25:53 dist-test-slave-f7th krb5kdc[4640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.235.194@KRBTEST.COM for oryx/127.22.235.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 08:25:53.280738 6470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:53.275502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:42715 (local address 127.22.235.194:54821)
0808 08:25:53.275768 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:53.275781 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:53.275878 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:53.276098 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:53.276099 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:53.276278 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:53.276371 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:53.276376 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:53.277220 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:53.277224 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:53.277958 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:53.277969 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:53.278392 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:53.278394 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:53.278452 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:53.278950 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:53.278962 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:53.280600 (+ 1638us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.235.254:42715: Server oryx/127.22.235.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":70,"threads_started":1}
W20260808 08:25:53.280893 6469 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:53.275568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:54821 (local address 127.22.235.254:42715)
0808 08:25:53.275734 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:53.275739 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:53.275843 (+ 104us) server_negotiation.cc:407] Connection header received
0808 08:25:53.275951 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:53.275954 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:53.276006 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:53.276073 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:53.276452 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:53.277112 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:53.278073 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:53.278303 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:53.278382 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:53.280769 (+ 2387us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.235.194:54821: BlockingRecv error: recv got EOF from 127.22.235.194:54821 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 08:25:53.379472 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 5644
I20260808 08:25:53.393023 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 5898
I20260808 08:25:53.407032 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 6138
I20260808 08:25:53.418375 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 5530
2026-08-08T08:25:53Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18176 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-08T08:25:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:25:53Z Disabled control of system clock
I20260808 08:25:53.459677 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:38763
--webserver_interface=127.22.235.254
--webserver_port=0
--builtin_ntp_servers=127.22.235.212:41021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:38763
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 08:25:53.569737 6476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:53.569983 6476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:53.570024 6476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:53.574230 6476 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:25:53.574281 6476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:53.574295 6476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:53.574311 6476 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:25:53.574323 6476 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:25:53.578966 6476 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:41021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:38763
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:38763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.235.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:53.579931 6476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:53.581270 6476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:53.589566 6490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:53.589566 6485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:53.589663 6476 server_base.cc:1034] running on GCE node
W20260808 08:25:53.589596 6486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:53.590319 6476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:53.591374 6476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:53.592527 6476 hybrid_clock.cc:648] HybridClock initialized: now 1786177553592504 us; error 35 us; skew 500 ppm
I20260808 08:25:53.597944 6476 webserver.cc:460] Webserver started at http://127.22.235.254:40023/ using document root <none> and password file <none>
I20260808 08:25:53.598491 6476 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:53.598549 6476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:53.598719 6476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:53.600732 6476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "1636e1ffeea24145b0ae3bb72fe150ae"
format_stamp: "Formatted at 2026-08-08 08:25:53 on dist-test-slave-f7th"
server_key: "ffd091d97c748f3445b91aea1cde6459"
server_key_iv: "d5b80acff0bf8d14baeadc3d7d61ee3d"
server_key_version: "encryptionkey@0"
I20260808 08:25:53.601246 6476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "1636e1ffeea24145b0ae3bb72fe150ae"
format_stamp: "Formatted at 2026-08-08 08:25:53 on dist-test-slave-f7th"
server_key: "ffd091d97c748f3445b91aea1cde6459"
server_key_iv: "d5b80acff0bf8d14baeadc3d7d61ee3d"
server_key_version: "encryptionkey@0"
I20260808 08:25:53.604700 6476 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 08:25:53.606995 6503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:53.608155 6476 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:25:53.608256 6476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "1636e1ffeea24145b0ae3bb72fe150ae"
format_stamp: "Formatted at 2026-08-08 08:25:53 on dist-test-slave-f7th"
server_key: "ffd091d97c748f3445b91aea1cde6459"
server_key_iv: "d5b80acff0bf8d14baeadc3d7d61ee3d"
server_key_version: "encryptionkey@0"
I20260808 08:25:53.608354 6476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:53.628580 6476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:53.629304 6476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:53.629480 6476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:53.636204 6476 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.254:38763
I20260808 08:25:53.636216 6608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.254:38763 every 8 connection(s)
I20260808 08:25:53.637140 6476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:25:53.640268 6609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:53.645148 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 6476
I20260808 08:25:53.645234 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:25:53.645462 23471 external_mini_cluster.cc:1428] Setting key d5fabbf3565ea51e6f9330c036f44e73
I20260808 08:25:53.645835 6609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1636e1ffeea24145b0ae3bb72fe150ae: Bootstrap starting.
I20260808 08:25:53.648293 6609 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1636e1ffeea24145b0ae3bb72fe150ae: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:53.649030 6609 log.cc:824] T 00000000000000000000000000000000 P 1636e1ffeea24145b0ae3bb72fe150ae: Log is configured to *not* fsync() on all Append() calls
I20260808 08:25:53.650803 6609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1636e1ffeea24145b0ae3bb72fe150ae: No bootstrap required, opened a new log
I20260808 08:25:53.654310 6615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:53.646844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55952 (local address 127.22.235.254:38763)
0808 08:25:53.647492 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:53.647509 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:53.647543 (+ 34us) server_negotiation.cc:407] Connection header received
0808 08:25:53.648141 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:53.648157 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:53.648556 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:53.648888 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:53.649299 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:53.650248 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:53.651170 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:53.651618 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:53.651761 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:53.652596 (+ 835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:53.652601 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:53.652639 (+ 38us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:25:53.653016 (+ 377us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:53.653070 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:25:53.653092 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:53.653174 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:53.653309 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 08:25:53.653557 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":138,"threads_started":1}
I20260808 08:25:53.654430 6609 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1636e1ffeea24145b0ae3bb72fe150ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1636e1ffeea24145b0ae3bb72fe150ae" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38763 } }
I20260808 08:25:53.654655 6609 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1636e1ffeea24145b0ae3bb72fe150ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:53.654711 6609 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1636e1ffeea24145b0ae3bb72fe150ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1636e1ffeea24145b0ae3bb72fe150ae, State: Initialized, Role: FOLLOWER
I20260808 08:25:53.655104 6609 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1636e1ffeea24145b0ae3bb72fe150ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1636e1ffeea24145b0ae3bb72fe150ae" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38763 } }
I20260808 08:25:53.655239 6609 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1636e1ffeea24145b0ae3bb72fe150ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:25:53.655334 6609 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1636e1ffeea24145b0ae3bb72fe150ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:25:53.655434 6609 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1636e1ffeea24145b0ae3bb72fe150ae [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:53.656493 6609 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1636e1ffeea24145b0ae3bb72fe150ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1636e1ffeea24145b0ae3bb72fe150ae" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38763 } }
I20260808 08:25:53.656843 6609 leader_election.cc:302] T 00000000000000000000000000000000 P 1636e1ffeea24145b0ae3bb72fe150ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1636e1ffeea24145b0ae3bb72fe150ae; no voters:
I20260808 08:25:53.657097 6609 leader_election.cc:288] T 00000000000000000000000000000000 P 1636e1ffeea24145b0ae3bb72fe150ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:25:53.657183 6618 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1636e1ffeea24145b0ae3bb72fe150ae [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:25:53.657377 6618 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1636e1ffeea24145b0ae3bb72fe150ae [term 1 LEADER]: Becoming Leader. State: Replica: 1636e1ffeea24145b0ae3bb72fe150ae, State: Running, Role: LEADER
I20260808 08:25:53.657732 6618 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1636e1ffeea24145b0ae3bb72fe150ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1636e1ffeea24145b0ae3bb72fe150ae" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38763 } }
I20260808 08:25:53.658239 6609 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1636e1ffeea24145b0ae3bb72fe150ae [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:25:53.659255 6622 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1636e1ffeea24145b0ae3bb72fe150ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1636e1ffeea24145b0ae3bb72fe150ae. Latest consensus state: current_term: 1 leader_uuid: "1636e1ffeea24145b0ae3bb72fe150ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1636e1ffeea24145b0ae3bb72fe150ae" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38763 } } }
I20260808 08:25:53.659379 6622 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1636e1ffeea24145b0ae3bb72fe150ae [sys.catalog]: This master's current role is: LEADER
I20260808 08:25:53.659647 6619 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1636e1ffeea24145b0ae3bb72fe150ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1636e1ffeea24145b0ae3bb72fe150ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1636e1ffeea24145b0ae3bb72fe150ae" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38763 } } }
I20260808 08:25:53.659726 6619 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1636e1ffeea24145b0ae3bb72fe150ae [sys.catalog]: This master's current role is: LEADER
I20260808 08:25:53.659750 6633 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:25:53.662796 6633 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:25:53.668156 6633 catalog_manager.cc:1347] Generated new cluster ID: 17cb2bab2f9048018efbe9eeaa98720f
I20260808 08:25:53.668219 6633 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:25:53.696568 6633 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:25:53.697513 6633 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:25:53.709290 6633 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1636e1ffeea24145b0ae3bb72fe150ae: Generated new TSK 0
I20260808 08:25:53.710044 6633 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 08:25:53.720923 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.193:0
--local_ip_for_outbound_sockets=127.22.235.193
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:38763
--builtin_ntp_servers=127.22.235.212:41021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 08:25:53.832420 6646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:53.832686 6646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:53.832738 6646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:53.836659 6646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:53.836712 6646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:53.836781 6646 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.193
I20260808 08:25:53.841388 6646 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:41021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:38763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.193
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:53.842404 6646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:53.843726 6646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:53.853101 6655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:53.853111 6653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:53.853190 6646 server_base.cc:1034] running on GCE node
W20260808 08:25:53.853096 6652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:53.853765 6646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:53.854437 6646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:53.855616 6646 hybrid_clock.cc:648] HybridClock initialized: now 1786177553855587 us; error 41 us; skew 500 ppm
I20260808 08:25:53.857559 6646 webserver.cc:460] Webserver started at http://127.22.235.193:37909/ using document root <none> and password file <none>
I20260808 08:25:53.858126 6646 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:53.858194 6646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:53.858409 6646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:53.860494 6646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/data/instance:
uuid: "2cc7b1e35b3a46f1ba26d57221b70e2d"
format_stamp: "Formatted at 2026-08-08 08:25:53 on dist-test-slave-f7th"
server_key: "ffa6d53917ba6edda2be77b8b5b3900d"
server_key_iv: "e1b9270ebfa5a0ee67d1af247b6a4c4d"
server_key_version: "encryptionkey@0"
I20260808 08:25:53.861085 6646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance:
uuid: "2cc7b1e35b3a46f1ba26d57221b70e2d"
format_stamp: "Formatted at 2026-08-08 08:25:53 on dist-test-slave-f7th"
server_key: "ffa6d53917ba6edda2be77b8b5b3900d"
server_key_iv: "e1b9270ebfa5a0ee67d1af247b6a4c4d"
server_key_version: "encryptionkey@0"
I20260808 08:25:53.864655 6646 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 08:25:53.867185 6666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:53.868357 6646 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:25:53.868467 6646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/wal
uuid: "2cc7b1e35b3a46f1ba26d57221b70e2d"
format_stamp: "Formatted at 2026-08-08 08:25:53 on dist-test-slave-f7th"
server_key: "ffa6d53917ba6edda2be77b8b5b3900d"
server_key_iv: "e1b9270ebfa5a0ee67d1af247b6a4c4d"
server_key_version: "encryptionkey@0"
I20260808 08:25:53.868561 6646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:53.888329 6646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:53.889077 6646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:53.889259 6646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:53.889883 6646 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:25:53.890863 6646 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:25:53.890913 6646 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:53.890969 6646 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:25:53.891000 6646 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:53.899334 6646 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.193:35029
I20260808 08:25:53.899394 6865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.193:35029 every 8 connection(s)
I20260808 08:25:53.900215 6646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
I20260808 08:25:53.907871 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 6646
I20260808 08:25:53.907977 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance
I20260808 08:25:53.908308 23471 external_mini_cluster.cc:1428] Setting key d58cff133d9044f788945d929f99ba27
I20260808 08:25:53.909168 6615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:53.901962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:58687 (local address 127.22.235.254:38763)
0808 08:25:53.902138 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:53.902143 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:53.903198 (+ 1055us) server_negotiation.cc:407] Connection header received
0808 08:25:53.904520 (+ 1322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:53.904523 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:53.904571 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:53.904684 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:53.905523 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:53.906102 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:53.907200 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:53.907428 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:53.907511 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:53.908580 (+ 1069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:53.908584 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:53.908587 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:25:53.908603 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:53.908660 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:25:53.908671 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:53.908747 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:53.908951 (+ 204us) server_negotiation.cc:299] Negotiation successful
0808 08:25:53.909028 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 08:25:53.910082 6871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:53.902396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:38763 (local address 127.22.235.193:58687)
0808 08:25:53.903059 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:53.903098 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:53.904315 (+ 1217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:53.904823 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:53.904830 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:53.904912 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:53.905384 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:53.905397 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:53.906197 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:53.906199 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:53.907085 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:53.907094 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:53.907548 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:53.907550 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:53.907700 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:53.908304 (+ 604us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 08:25:53.908306 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:53.908346 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 08:25:53.908425 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:53.908752 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:53.908760 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:53.908766 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 08:25:53.908896 (+ 130us) client_negotiation.cc:240] Negotiation successful
0808 08:25:53.909147 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":125,"threads_started":1}
I20260808 08:25:53.910380 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.194:0
--local_ip_for_outbound_sockets=127.22.235.194
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:38763
--builtin_ntp_servers=127.22.235.212:41021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 08:25:53.911213 6866 heartbeater.cc:347] Connected to a master server at 127.22.235.254:38763
I20260808 08:25:53.911451 6866 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:53.912055 6866 heartbeater.cc:511] Master 127.22.235.254:38763 requested a full tablet report, sending...
I20260808 08:25:53.913591 6543 ts_manager.cc:194] Registered new tserver with Master: 2cc7b1e35b3a46f1ba26d57221b70e2d (127.22.235.193:35029)
I20260808 08:25:53.915228 6543 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.235.193:58687
W20260808 08:25:54.024778 6872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:54.025018 6872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:54.025051 6872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:54.028963 6872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:54.029013 6872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:54.029062 6872 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.194
I20260808 08:25:54.033548 6872 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:41021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:38763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6872
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.194
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:54.034466 6872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:54.035694 6872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:54.044674 6885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:54.044697 6887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:54.044826 6872 server_base.cc:1034] running on GCE node
W20260808 08:25:54.044674 6882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:54.045358 6872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:54.045969 6872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:54.047119 6872 hybrid_clock.cc:648] HybridClock initialized: now 1786177554047094 us; error 38 us; skew 500 ppm
I20260808 08:25:54.049057 6872 webserver.cc:460] Webserver started at http://127.22.235.194:33551/ using document root <none> and password file <none>
I20260808 08:25:54.049615 6872 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:54.049674 6872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:54.049868 6872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:54.051818 6872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/data/instance:
uuid: "9fc8aa3214fa4292b4ec817ee5e25540"
format_stamp: "Formatted at 2026-08-08 08:25:54 on dist-test-slave-f7th"
server_key: "fca879a04cf4d7f4b891f6b05cbd5070"
server_key_iv: "bc4af258f4146b14bf4968e17be7ed90"
server_key_version: "encryptionkey@0"
I20260808 08:25:54.052342 6872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance:
uuid: "9fc8aa3214fa4292b4ec817ee5e25540"
format_stamp: "Formatted at 2026-08-08 08:25:54 on dist-test-slave-f7th"
server_key: "fca879a04cf4d7f4b891f6b05cbd5070"
server_key_iv: "bc4af258f4146b14bf4968e17be7ed90"
server_key_version: "encryptionkey@0"
I20260808 08:25:54.055932 6872 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 08:25:54.058454 6894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:54.059684 6872 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:25:54.059779 6872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/wal
uuid: "9fc8aa3214fa4292b4ec817ee5e25540"
format_stamp: "Formatted at 2026-08-08 08:25:54 on dist-test-slave-f7th"
server_key: "fca879a04cf4d7f4b891f6b05cbd5070"
server_key_iv: "bc4af258f4146b14bf4968e17be7ed90"
server_key_version: "encryptionkey@0"
I20260808 08:25:54.059880 6872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:54.075367 6872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:54.076042 6872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:54.076206 6872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:54.076822 6872 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:25:54.077775 6872 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:25:54.077834 6872 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:54.077893 6872 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:25:54.077924 6872 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:54.086231 6872 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.194:44171
I20260808 08:25:54.086262 7095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.194:44171 every 8 connection(s)
I20260808 08:25:54.087092 6872 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
I20260808 08:25:54.087984 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 6872
I20260808 08:25:54.088096 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance
I20260808 08:25:54.088405 23471 external_mini_cluster.cc:1428] Setting key d682538a66defdde92bbdc9a76977a5a
I20260808 08:25:54.090484 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.195:0
--local_ip_for_outbound_sockets=127.22.235.195
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:38763
--builtin_ntp_servers=127.22.235.212:41021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 08:25:54.097265 6615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:54.088997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:48037 (local address 127.22.235.254:38763)
0808 08:25:54.089142 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:54.089147 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:54.090314 (+ 1167us) server_negotiation.cc:407] Connection header received
0808 08:25:54.092014 (+ 1700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:54.092016 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:54.092062 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:54.092127 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:54.093082 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:54.093794 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:54.095359 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:54.095573 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:54.095646 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:54.096834 (+ 1188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:54.096837 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:54.096839 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:25:54.096852 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:54.096871 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:25:54.096878 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:54.096932 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:54.097105 (+ 173us) server_negotiation.cc:299] Negotiation successful
0808 08:25:54.097152 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 08:25:54.098452 7098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:54.089423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:38763 (local address 127.22.235.194:48037)
0808 08:25:54.090154 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:54.090208 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:54.091785 (+ 1577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:54.092321 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:54.092331 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:54.092432 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:54.092940 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:54.092953 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:54.093917 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:54.093921 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:54.095246 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:54.095259 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:54.095718 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:54.095723 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:54.095920 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:54.096518 (+ 598us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 08:25:54.096521 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:54.096576 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 08:25:54.096690 (+ 114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:54.096952 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:54.096961 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:54.096975 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 08:25:54.097106 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 08:25:54.097408 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":114,"threads_started":1}
I20260808 08:25:54.099617 7096 heartbeater.cc:347] Connected to a master server at 127.22.235.254:38763
I20260808 08:25:54.099881 7096 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:54.100533 7096 heartbeater.cc:511] Master 127.22.235.254:38763 requested a full tablet report, sending...
I20260808 08:25:54.101728 6543 ts_manager.cc:194] Registered new tserver with Master: 9fc8aa3214fa4292b4ec817ee5e25540 (127.22.235.194:44171)
I20260808 08:25:54.102500 6543 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.235.194:48037
W20260808 08:25:54.205478 7100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:54.205713 7100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:54.205746 7100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:54.209692 7100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:54.209745 7100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:54.209794 7100 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.195
I20260808 08:25:54.214412 7100 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:41021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:38763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7100
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.195
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:54.215351 7100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:54.216640 7100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:54.225822 7108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:54.225777 7112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:54.225798 7107 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:54.225979 7100 server_base.cc:1034] running on GCE node
I20260808 08:25:54.226467 7100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:54.227109 7100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:54.228260 7100 hybrid_clock.cc:648] HybridClock initialized: now 1786177554228247 us; error 32 us; skew 500 ppm
I20260808 08:25:54.229983 7100 webserver.cc:460] Webserver started at http://127.22.235.195:34301/ using document root <none> and password file <none>
I20260808 08:25:54.230520 7100 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:54.230594 7100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:54.230800 7100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:54.232810 7100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/data/instance:
uuid: "5802150871ad4cac80a88cc800ead05d"
format_stamp: "Formatted at 2026-08-08 08:25:54 on dist-test-slave-f7th"
server_key: "50d178a7dd5689e524611db9bfa2cc81"
server_key_iv: "e384020b0838b9e157eeadd5b5b98cf5"
server_key_version: "encryptionkey@0"
I20260808 08:25:54.233320 7100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance:
uuid: "5802150871ad4cac80a88cc800ead05d"
format_stamp: "Formatted at 2026-08-08 08:25:54 on dist-test-slave-f7th"
server_key: "50d178a7dd5689e524611db9bfa2cc81"
server_key_iv: "e384020b0838b9e157eeadd5b5b98cf5"
server_key_version: "encryptionkey@0"
I20260808 08:25:54.236728 7100 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 08:25:54.239025 7125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:54.240142 7100 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:25:54.240257 7100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/wal
uuid: "5802150871ad4cac80a88cc800ead05d"
format_stamp: "Formatted at 2026-08-08 08:25:54 on dist-test-slave-f7th"
server_key: "50d178a7dd5689e524611db9bfa2cc81"
server_key_iv: "e384020b0838b9e157eeadd5b5b98cf5"
server_key_version: "encryptionkey@0"
I20260808 08:25:54.240347 7100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:54.259258 7100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:54.259965 7100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:54.260160 7100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:54.260811 7100 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:25:54.261768 7100 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:25:54.261816 7100 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:54.261870 7100 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:25:54.261899 7100 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:54.270212 7100 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.195:42355
I20260808 08:25:54.270263 7328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.195:42355 every 8 connection(s)
I20260808 08:25:54.271091 7100 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
I20260808 08:25:54.277679 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 7100
I20260808 08:25:54.277786 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance
I20260808 08:25:54.278064 23471 external_mini_cluster.cc:1428] Setting key 7afb528df77ca3cf0e4b37939588e6ab
I20260808 08:25:54.280231 6615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:54.272938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:37589 (local address 127.22.235.254:38763)
0808 08:25:54.273081 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:54.273085 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:54.274128 (+ 1043us) server_negotiation.cc:407] Connection header received
0808 08:25:54.275445 (+ 1317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:54.275448 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:54.275501 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:54.275571 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:54.276401 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:54.276935 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:54.278258 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:54.278522 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:54.278614 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:54.279760 (+ 1146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:54.279764 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:54.279767 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:25:54.279780 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:54.279802 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:25:54.279813 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:54.279887 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:54.280034 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 08:25:54.280097 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 08:25:54.281076 7332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:54.273339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:38763 (local address 127.22.235.195:37589)
0808 08:25:54.274003 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:54.274045 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:54.275270 (+ 1225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:54.275717 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:54.275724 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:54.275806 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:54.276295 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:54.276307 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:54.277024 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:54.277027 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:54.278144 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:54.278157 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:54.278685 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:54.278689 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:54.278862 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:54.279509 (+ 647us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 08:25:54.279512 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:54.279555 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 08:25:54.279618 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:54.279886 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:54.279891 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:54.279895 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 08:25:54.279983 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 08:25:54.280244 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":127,"threads_started":1}
I20260808 08:25:54.282071 7329 heartbeater.cc:347] Connected to a master server at 127.22.235.254:38763
I20260808 08:25:54.282280 7329 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:54.282783 7329 heartbeater.cc:511] Master 127.22.235.254:38763 requested a full tablet report, sending...
I20260808 08:25:54.283766 6543 ts_manager.cc:194] Registered new tserver with Master: 5802150871ad4cac80a88cc800ead05d (127.22.235.195:42355)
I20260808 08:25:54.284330 6543 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.235.195:37589
I20260808 08:25:54.291852 23471 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 08:25:54.294549 6615 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:54.293756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55956 (local address 127.22.235.254:38763)
0808 08:25:54.293870 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:54.293874 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:54.294029 (+ 155us) server_negotiation.cc:407] Connection header received
0808 08:25:54.294132 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:54.294135 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:54.294174 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:54.294236 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:54.294443 (+ 207us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55956: BlockingRecv error: recv got EOF from 127.0.0.1:55956 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 08:25:54.294739 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 6646
I20260808 08:25:54.315531 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 6872
I20260808 08:25:54.347414 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 7100
I20260808 08:25:54.361296 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 6476
2026-08-08T08:25:54Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (938 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-08T08:25:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:25:54Z Disabled control of system clock
I20260808 08:25:54.397171 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:38293
--webserver_interface=127.22.235.254
--webserver_port=0
--builtin_ntp_servers=127.22.235.212:46703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:38293
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 08:25:54.508214 7351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:54.508457 7351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:54.508491 7351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:54.512416 7351 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:25:54.512470 7351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:54.512485 7351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:54.512499 7351 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:25:54.512512 7351 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:25:54.517122 7351 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:46703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:38293
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:38293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.235.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:54.518117 7351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:54.519447 7351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:54.527629 7364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:54.527671 7360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:54.527722 7366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:54.527817 7351 server_base.cc:1034] running on GCE node
I20260808 08:25:54.528494 7351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:54.529670 7351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:54.530858 7351 hybrid_clock.cc:648] HybridClock initialized: now 1786177554530839 us; error 27 us; skew 500 ppm
I20260808 08:25:54.532829 7351 webserver.cc:460] Webserver started at http://127.22.235.254:38209/ using document root <none> and password file <none>
I20260808 08:25:54.533561 7351 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:54.533653 7351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:54.533874 7351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:54.535836 7351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "0f4839440f09477fb872b099620ee230"
format_stamp: "Formatted at 2026-08-08 08:25:54 on dist-test-slave-f7th"
server_key: "001ba26e2f7b7296e39c571060f1a727"
server_key_iv: "a111955f93afea0ffc75f9b1193f400e"
server_key_version: "encryptionkey@0"
I20260808 08:25:54.536391 7351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "0f4839440f09477fb872b099620ee230"
format_stamp: "Formatted at 2026-08-08 08:25:54 on dist-test-slave-f7th"
server_key: "001ba26e2f7b7296e39c571060f1a727"
server_key_iv: "a111955f93afea0ffc75f9b1193f400e"
server_key_version: "encryptionkey@0"
I20260808 08:25:54.539963 7351 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260808 08:25:54.542340 7374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:54.543521 7351 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:25:54.543643 7351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "0f4839440f09477fb872b099620ee230"
format_stamp: "Formatted at 2026-08-08 08:25:54 on dist-test-slave-f7th"
server_key: "001ba26e2f7b7296e39c571060f1a727"
server_key_iv: "a111955f93afea0ffc75f9b1193f400e"
server_key_version: "encryptionkey@0"
I20260808 08:25:54.543738 7351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:54.548199 7351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:54.548804 7351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:54.548961 7351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:54.555231 7351 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.254:38293
I20260808 08:25:54.555248 7452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.254:38293 every 8 connection(s)
I20260808 08:25:54.556130 7351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:25:54.559284 7454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:54.562448 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 7351
I20260808 08:25:54.562574 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:25:54.562875 23471 external_mini_cluster.cc:1428] Setting key 2a318844055158bcc9b67d3a4adb8d0d
I20260808 08:25:54.565300 7454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f4839440f09477fb872b099620ee230: Bootstrap starting.
I20260808 08:25:54.567592 7454 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f4839440f09477fb872b099620ee230: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:54.568373 7454 log.cc:824] T 00000000000000000000000000000000 P 0f4839440f09477fb872b099620ee230: Log is configured to *not* fsync() on all Append() calls
I20260808 08:25:54.568495 7466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:54.564325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51414 (local address 127.22.235.254:38293)
0808 08:25:54.565026 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:54.565043 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:54.565080 (+ 37us) server_negotiation.cc:407] Connection header received
0808 08:25:54.565727 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:54.565746 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:54.566096 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:54.566424 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:54.567234 (+ 810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:54.567240 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:54.567252 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:25:54.567263 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:54.567319 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:25:54.567335 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:54.567433 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:54.567593 (+ 160us) server_negotiation.cc:299] Negotiation successful
0808 08:25:54.567833 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":132,"threads_started":1}
I20260808 08:25:54.570338 7454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f4839440f09477fb872b099620ee230: No bootstrap required, opened a new log
I20260808 08:25:54.573761 7454 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f4839440f09477fb872b099620ee230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f4839440f09477fb872b099620ee230" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38293 } }
I20260808 08:25:54.573999 7454 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0f4839440f09477fb872b099620ee230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:54.574059 7454 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f4839440f09477fb872b099620ee230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f4839440f09477fb872b099620ee230, State: Initialized, Role: FOLLOWER
I20260808 08:25:54.574436 7454 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f4839440f09477fb872b099620ee230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f4839440f09477fb872b099620ee230" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38293 } }
I20260808 08:25:54.574564 7454 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f4839440f09477fb872b099620ee230 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:25:54.574628 7454 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f4839440f09477fb872b099620ee230 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:25:54.574728 7454 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f4839440f09477fb872b099620ee230 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:54.575697 7454 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f4839440f09477fb872b099620ee230 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f4839440f09477fb872b099620ee230" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38293 } }
I20260808 08:25:54.576025 7454 leader_election.cc:302] T 00000000000000000000000000000000 P 0f4839440f09477fb872b099620ee230 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f4839440f09477fb872b099620ee230; no voters:
I20260808 08:25:54.576296 7454 leader_election.cc:288] T 00000000000000000000000000000000 P 0f4839440f09477fb872b099620ee230 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:25:54.576382 7469 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f4839440f09477fb872b099620ee230 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:25:54.576637 7469 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f4839440f09477fb872b099620ee230 [term 1 LEADER]: Becoming Leader. State: Replica: 0f4839440f09477fb872b099620ee230, State: Running, Role: LEADER
I20260808 08:25:54.576968 7469 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f4839440f09477fb872b099620ee230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f4839440f09477fb872b099620ee230" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38293 } }
I20260808 08:25:54.577545 7454 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f4839440f09477fb872b099620ee230 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:25:54.578663 7473 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f4839440f09477fb872b099620ee230 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f4839440f09477fb872b099620ee230. Latest consensus state: current_term: 1 leader_uuid: "0f4839440f09477fb872b099620ee230" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f4839440f09477fb872b099620ee230" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38293 } } }
I20260808 08:25:54.578647 7472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f4839440f09477fb872b099620ee230 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0f4839440f09477fb872b099620ee230" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f4839440f09477fb872b099620ee230" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38293 } } }
I20260808 08:25:54.578789 7472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f4839440f09477fb872b099620ee230 [sys.catalog]: This master's current role is: LEADER
I20260808 08:25:54.578789 7473 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f4839440f09477fb872b099620ee230 [sys.catalog]: This master's current role is: LEADER
I20260808 08:25:54.579171 7487 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:25:54.582486 7487 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:25:54.587972 7487 catalog_manager.cc:1347] Generated new cluster ID: 6effd26cca6b40ba9699db9624fa0539
I20260808 08:25:54.588028 7487 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:25:54.605948 7487 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:25:54.606245 7487 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:25:54.616554 7487 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f4839440f09477fb872b099620ee230: Generated new TSK 0
I20260808 08:25:54.617262 7487 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 08:25:54.628209 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.193:0
--local_ip_for_outbound_sockets=127.22.235.193
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:38293
--builtin_ntp_servers=127.22.235.212:46703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 08:25:54.763629 7506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:54.763871 7506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:54.763912 7506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:54.767913 7506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:54.767968 7506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:54.768046 7506 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.193
I20260808 08:25:54.772634 7506 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:46703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:38293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7506
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.193
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:54.773622 7506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:54.775002 7506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:54.784343 7514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:54.784374 7515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:54.784365 7517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:54.784557 7506 server_base.cc:1034] running on GCE node
I20260808 08:25:54.785090 7506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:54.785743 7506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:54.786901 7506 hybrid_clock.cc:648] HybridClock initialized: now 1786177554786881 us; error 31 us; skew 500 ppm
I20260808 08:25:54.788755 7506 webserver.cc:460] Webserver started at http://127.22.235.193:44791/ using document root <none> and password file <none>
I20260808 08:25:54.789314 7506 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:54.789381 7506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:54.789588 7506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:54.791610 7506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/data/instance:
uuid: "d8e349af8efd4e11b6f001feadcbe2f5"
format_stamp: "Formatted at 2026-08-08 08:25:54 on dist-test-slave-f7th"
server_key: "aa94ea64df8b801dad7a2ccc92c8bde4"
server_key_iv: "e515a9c3054a7d13ef7425334ae894c4"
server_key_version: "encryptionkey@0"
I20260808 08:25:54.792150 7506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance:
uuid: "d8e349af8efd4e11b6f001feadcbe2f5"
format_stamp: "Formatted at 2026-08-08 08:25:54 on dist-test-slave-f7th"
server_key: "aa94ea64df8b801dad7a2ccc92c8bde4"
server_key_iv: "e515a9c3054a7d13ef7425334ae894c4"
server_key_version: "encryptionkey@0"
I20260808 08:25:54.795770 7506 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 08:25:54.798331 7530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:54.799502 7506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 08:25:54.799611 7506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/wal
uuid: "d8e349af8efd4e11b6f001feadcbe2f5"
format_stamp: "Formatted at 2026-08-08 08:25:54 on dist-test-slave-f7th"
server_key: "aa94ea64df8b801dad7a2ccc92c8bde4"
server_key_iv: "e515a9c3054a7d13ef7425334ae894c4"
server_key_version: "encryptionkey@0"
I20260808 08:25:54.799711 7506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:54.804126 7506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:54.804728 7506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:54.804883 7506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:54.805408 7506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:25:54.806305 7506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:25:54.806352 7506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:54.806402 7506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:25:54.806432 7506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:54.814213 7506 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.193:38719
I20260808 08:25:54.814266 7725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.193:38719 every 8 connection(s)
I20260808 08:25:54.815058 7506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
I20260808 08:25:54.815673 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 7506
I20260808 08:25:54.815789 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance
I20260808 08:25:54.816138 23471 external_mini_cluster.cc:1428] Setting key 80bec04ef5a1aa37875006e6b8e297ce
I20260808 08:25:54.818473 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.194:0
--local_ip_for_outbound_sockets=127.22.235.194
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:38293
--builtin_ntp_servers=127.22.235.212:46703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 08:25:54.821880 7466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:54.817056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:51087 (local address 127.22.235.254:38293)
0808 08:25:54.817620 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:54.817624 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:54.818407 (+ 783us) server_negotiation.cc:407] Connection header received
0808 08:25:54.820089 (+ 1682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:54.820091 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:54.820129 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:54.820200 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:54.821387 (+ 1187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:54.821391 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:54.821394 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:25:54.821408 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:54.821432 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:25:54.821441 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:54.821557 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:54.821680 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 08:25:54.821732 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":488}
I20260808 08:25:54.822741 7731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:54.817441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:38293 (local address 127.22.235.193:51087)
0808 08:25:54.818244 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:54.818311 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:54.819860 (+ 1549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:54.820408 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:54.820418 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:54.820523 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:54.821115 (+ 592us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 08:25:54.821117 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:54.821158 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 08:25:54.821231 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:54.821537 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:54.821542 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:54.821546 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 08:25:54.821696 (+ 150us) client_negotiation.cc:240] Negotiation successful
0808 08:25:54.821943 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":125,"threads_started":1}
I20260808 08:25:54.824213 7727 heartbeater.cc:347] Connected to a master server at 127.22.235.254:38293
I20260808 08:25:54.824568 7727 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:54.825013 7727 heartbeater.cc:511] Master 127.22.235.254:38293 requested a full tablet report, sending...
I20260808 08:25:54.826759 7401 ts_manager.cc:194] Registered new tserver with Master: d8e349af8efd4e11b6f001feadcbe2f5 (127.22.235.193:38719)
W20260808 08:25:54.932916 7732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:54.933174 7732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:54.933212 7732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:54.937151 7732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:54.937204 7732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:54.937258 7732 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.194
I20260808 08:25:54.941913 7732 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:46703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:38293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7732
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.194
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:54.942945 7732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:54.944257 7732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:54.953637 7739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:54.953728 7732 server_base.cc:1034] running on GCE node
W20260808 08:25:54.953655 7737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:54.953637 7742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:54.954273 7732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:54.954895 7732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:54.956064 7732 hybrid_clock.cc:648] HybridClock initialized: now 1786177554956046 us; error 27 us; skew 500 ppm
I20260808 08:25:54.957794 7732 webserver.cc:460] Webserver started at http://127.22.235.194:42635/ using document root <none> and password file <none>
I20260808 08:25:54.958314 7732 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:54.958371 7732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:54.958539 7732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:54.960433 7732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/data/instance:
uuid: "cf5c0de1097e4ef9a71ca61754777d3a"
format_stamp: "Formatted at 2026-08-08 08:25:54 on dist-test-slave-f7th"
server_key: "8d46cd6642ea22d0cbed88e6b822176c"
server_key_iv: "0af64e3eff0396b3c686668a82a81ea5"
server_key_version: "encryptionkey@0"
I20260808 08:25:54.960960 7732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance:
uuid: "cf5c0de1097e4ef9a71ca61754777d3a"
format_stamp: "Formatted at 2026-08-08 08:25:54 on dist-test-slave-f7th"
server_key: "8d46cd6642ea22d0cbed88e6b822176c"
server_key_iv: "0af64e3eff0396b3c686668a82a81ea5"
server_key_version: "encryptionkey@0"
I20260808 08:25:54.964367 7732 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 08:25:54.966722 7755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:54.967849 7732 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:25:54.967952 7732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/wal
uuid: "cf5c0de1097e4ef9a71ca61754777d3a"
format_stamp: "Formatted at 2026-08-08 08:25:54 on dist-test-slave-f7th"
server_key: "8d46cd6642ea22d0cbed88e6b822176c"
server_key_iv: "0af64e3eff0396b3c686668a82a81ea5"
server_key_version: "encryptionkey@0"
I20260808 08:25:54.968047 7732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:54.972291 7732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:54.972872 7732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:54.973024 7732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:54.973520 7732 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:25:54.974355 7732 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:25:54.974401 7732 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:54.974449 7732 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:25:54.974475 7732 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:54.982713 7732 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.194:43715
I20260808 08:25:54.982736 7964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.194:43715 every 8 connection(s)
I20260808 08:25:54.983523 7732 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
I20260808 08:25:54.985414 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 7732
I20260808 08:25:54.985532 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance
I20260808 08:25:54.985838 23471 external_mini_cluster.cc:1428] Setting key a76ce74c68c008fae1c7a2cc92083d46
I20260808 08:25:54.987780 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.195:0
--local_ip_for_outbound_sockets=127.22.235.195
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:38293
--builtin_ntp_servers=127.22.235.212:46703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 08:25:54.989406 7466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:54.985213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:45027 (local address 127.22.235.254:38293)
0808 08:25:54.985345 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:54.985348 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:54.986526 (+ 1178us) server_negotiation.cc:407] Connection header received
0808 08:25:54.987842 (+ 1316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:54.987845 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:54.987886 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:54.987959 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:54.988989 (+ 1030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:54.988993 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:54.988996 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:25:54.989007 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:54.989028 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:25:54.989036 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:54.989108 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:54.989234 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 08:25:54.989278 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 08:25:54.990410 7974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:54.985711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:38293 (local address 127.22.235.194:45027)
0808 08:25:54.986384 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:54.986420 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:54.987647 (+ 1227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:54.988114 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:54.988121 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:54.988210 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:54.988745 (+ 535us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 08:25:54.988746 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:54.988783 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 08:25:54.988852 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:54.989104 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:54.989113 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:54.989119 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 08:25:54.989266 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 08:25:54.989498 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":139,"threads_started":1}
I20260808 08:25:54.991537 7971 heartbeater.cc:347] Connected to a master server at 127.22.235.254:38293
I20260808 08:25:54.991751 7971 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:54.992131 7971 heartbeater.cc:511] Master 127.22.235.254:38293 requested a full tablet report, sending...
I20260808 08:25:54.993167 7401 ts_manager.cc:194] Registered new tserver with Master: cf5c0de1097e4ef9a71ca61754777d3a (127.22.235.194:43715)
W20260808 08:25:55.100955 7976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:55.101220 7976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:55.101258 7976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:55.105307 7976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:55.105360 7976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:55.105412 7976 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.195
I20260808 08:25:55.110065 7976 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:46703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:38293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7976
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.195
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:55.111056 7976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:55.112366 7976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:55.121747 7984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:55.121790 7976 server_base.cc:1034] running on GCE node
W20260808 08:25:55.121753 7988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:55.121753 7990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:55.122408 7976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:55.123025 7976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:55.124171 7976 hybrid_clock.cc:648] HybridClock initialized: now 1786177555124144 us; error 38 us; skew 500 ppm
I20260808 08:25:55.125995 7976 webserver.cc:460] Webserver started at http://127.22.235.195:38877/ using document root <none> and password file <none>
I20260808 08:25:55.126554 7976 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:55.126616 7976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:55.126835 7976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:55.128868 7976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/data/instance:
uuid: "ebd84835385742eabe769d4bfa6a3fff"
format_stamp: "Formatted at 2026-08-08 08:25:55 on dist-test-slave-f7th"
server_key: "d4590e03e7e8ad5ed946fdec929124c5"
server_key_iv: "8b23c7a13f109885b5c9824beff05809"
server_key_version: "encryptionkey@0"
I20260808 08:25:55.129395 7976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance:
uuid: "ebd84835385742eabe769d4bfa6a3fff"
format_stamp: "Formatted at 2026-08-08 08:25:55 on dist-test-slave-f7th"
server_key: "d4590e03e7e8ad5ed946fdec929124c5"
server_key_iv: "8b23c7a13f109885b5c9824beff05809"
server_key_version: "encryptionkey@0"
I20260808 08:25:55.132898 7976 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 08:25:55.135303 8000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:55.136775 7976 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:25:55.136871 7976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/wal
uuid: "ebd84835385742eabe769d4bfa6a3fff"
format_stamp: "Formatted at 2026-08-08 08:25:55 on dist-test-slave-f7th"
server_key: "d4590e03e7e8ad5ed946fdec929124c5"
server_key_iv: "8b23c7a13f109885b5c9824beff05809"
server_key_version: "encryptionkey@0"
I20260808 08:25:55.136955 7976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:55.141224 7976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:55.141755 7976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:55.141889 7976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:55.142381 7976 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:25:55.143219 7976 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:25:55.143265 7976 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:55.143301 7976 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:25:55.143321 7976 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:55.151861 7976 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.195:42477
I20260808 08:25:55.151880 8201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.195:42477 every 8 connection(s)
I20260808 08:25:55.152823 7976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
I20260808 08:25:55.155268 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 7976
I20260808 08:25:55.155367 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance
I20260808 08:25:55.155646 23471 external_mini_cluster.cc:1428] Setting key fe732429cdc28774f36cd7c6b8bb0eef
I20260808 08:25:55.158711 7466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:55.154499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:47445 (local address 127.22.235.254:38293)
0808 08:25:55.154637 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:55.154641 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:55.155792 (+ 1151us) server_negotiation.cc:407] Connection header received
0808 08:25:55.157180 (+ 1388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:55.157183 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:55.157224 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:55.157312 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:55.158378 (+ 1066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:55.158381 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:55.158383 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:25:55.158394 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:55.158414 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:25:55.158422 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:55.158492 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:55.158579 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 08:25:55.158612 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 08:25:55.159520 8206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:55.154891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:38293 (local address 127.22.235.195:47445)
0808 08:25:55.155617 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:55.155668 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:55.157000 (+ 1332us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:55.157487 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:55.157496 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:55.157595 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:55.158153 (+ 558us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 08:25:55.158154 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:55.158194 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 08:25:55.158262 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:55.158485 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:55.158490 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:55.158494 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 08:25:55.158615 (+ 121us) client_negotiation.cc:240] Negotiation successful
0808 08:25:55.158800 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":109,"threads_started":1}
I20260808 08:25:55.160456 8202 heartbeater.cc:347] Connected to a master server at 127.22.235.254:38293
I20260808 08:25:55.160686 8202 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:55.161074 8202 heartbeater.cc:511] Master 127.22.235.254:38293 requested a full tablet report, sending...
I20260808 08:25:55.161988 7401 ts_manager.cc:194] Registered new tserver with Master: ebd84835385742eabe769d4bfa6a3fff (127.22.235.195:42477)
I20260808 08:25:55.169345 23471 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 08:25:55.172058 7466 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:25:55.171165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51418 (local address 127.22.235.254:38293)
0808 08:25:55.171275 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:55.171279 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:55.171448 (+ 169us) server_negotiation.cc:407] Connection header received
0808 08:25:55.171601 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:55.171604 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:55.171639 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:55.171702 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:55.171925 (+ 223us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51418: BlockingRecv error: recv got EOF from 127.0.0.1:51418 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 08:25:55.172154 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 7506
I20260808 08:25:55.191188 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 7732
I20260808 08:25:55.202653 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 7976
I20260808 08:25:55.216183 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 7351
2026-08-08T08:25:55Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (856 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 08:25:55 dist-test-slave-f7th krb5kdc[8222](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 08:25:55 dist-test-slave-f7th krb5kdc[8222](info): set up 2 sockets
Aug 08 08:25:55 dist-test-slave-f7th krb5kdc[8222](info): commencing operation
krb5kdc: starting...
W20260808 08:25:56.379951 23471 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Aug 08 08:25:56 dist-test-slave-f7th krb5kdc[8222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177556, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T08:25:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:25:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.235.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
I20260808 08:25:56.497635 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:44295
--webserver_interface=127.22.235.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:40327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:44295
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:25:56.608539 8250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:56.608839 8250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:56.608884 8250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:56.612831 8250 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:25:56.612890 8250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:56.612907 8250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:56.612936 8250 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:25:56.612964 8250 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:25:56.618153 8250 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:40327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:44295
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:44295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.235.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:56.619186 8250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:56.620185 8250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:56.628427 8258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:56.628497 8257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:56.628407 8264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:56.628715 8250 server_base.cc:1034] running on GCE node
I20260808 08:25:56.629254 8250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:56.630322 8250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:56.631475 8250 hybrid_clock.cc:648] HybridClock initialized: now 1786177556631453 us; error 36 us; skew 500 ppm
Aug 08 08:25:56 dist-test-slave-f7th krb5kdc[8222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177556, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:56.634200 8250 init.cc:377] Logged in from keytab as kudu/127.22.235.254@KRBTEST.COM (short username kudu)
I20260808 08:25:56.635064 8250 webserver.cc:460] Webserver started at http://127.22.235.254:35669/ using document root <none> and password file <none>
I20260808 08:25:56.635576 8250 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:56.635632 8250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:56.635804 8250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:56.637782 8250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "5de71d9793b4459e8e0fda5237f5cfa0"
format_stamp: "Formatted at 2026-08-08 08:25:56 on dist-test-slave-f7th"
server_key: "c7fdd65f22e41cc3121639e098366bcb"
server_key_iv: "543c4ad31cfc201de71c2461d52b84af"
server_key_version: "encryptionkey@0"
I20260808 08:25:56.638289 8250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "5de71d9793b4459e8e0fda5237f5cfa0"
format_stamp: "Formatted at 2026-08-08 08:25:56 on dist-test-slave-f7th"
server_key: "c7fdd65f22e41cc3121639e098366bcb"
server_key_iv: "543c4ad31cfc201de71c2461d52b84af"
server_key_version: "encryptionkey@0"
I20260808 08:25:56.641757 8250 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 08:25:56.643980 8275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:56.645169 8250 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:25:56.645288 8250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "5de71d9793b4459e8e0fda5237f5cfa0"
format_stamp: "Formatted at 2026-08-08 08:25:56 on dist-test-slave-f7th"
server_key: "c7fdd65f22e41cc3121639e098366bcb"
server_key_iv: "543c4ad31cfc201de71c2461d52b84af"
server_key_version: "encryptionkey@0"
I20260808 08:25:56.645378 8250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:56.664449 8250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:56.667514 8250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:56.667687 8250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:56.674638 8250 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.254:44295
I20260808 08:25:56.674661 8378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.254:44295 every 8 connection(s)
I20260808 08:25:56.675537 8250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:25:56.678246 8379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:56.683633 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 8250
I20260808 08:25:56.683756 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:25:56.684059 23471 external_mini_cluster.cc:1428] Setting key edd7fc7508ce36e9383c13cab21c41e1
I20260808 08:25:56.684047 8379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5de71d9793b4459e8e0fda5237f5cfa0: Bootstrap starting.
I20260808 08:25:56.686645 8379 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5de71d9793b4459e8e0fda5237f5cfa0: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:56.687542 8379 log.cc:824] T 00000000000000000000000000000000 P 5de71d9793b4459e8e0fda5237f5cfa0: Log is configured to *not* fsync() on all Append() calls
I20260808 08:25:56.689702 8379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5de71d9793b4459e8e0fda5237f5cfa0: No bootstrap required, opened a new log
Aug 08 08:25:56 dist-test-slave-f7th krb5kdc[8222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177556, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:25:56.693917 8379 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5de71d9793b4459e8e0fda5237f5cfa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5de71d9793b4459e8e0fda5237f5cfa0" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 44295 } }
I20260808 08:25:56.694144 8379 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5de71d9793b4459e8e0fda5237f5cfa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:56.694213 8379 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5de71d9793b4459e8e0fda5237f5cfa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5de71d9793b4459e8e0fda5237f5cfa0, State: Initialized, Role: FOLLOWER
I20260808 08:25:56.694720 8379 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5de71d9793b4459e8e0fda5237f5cfa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5de71d9793b4459e8e0fda5237f5cfa0" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 44295 } }
I20260808 08:25:56.694849 8379 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5de71d9793b4459e8e0fda5237f5cfa0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:25:56.694912 8379 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5de71d9793b4459e8e0fda5237f5cfa0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:25:56.695017 8379 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5de71d9793b4459e8e0fda5237f5cfa0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:56.695823 8379 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5de71d9793b4459e8e0fda5237f5cfa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5de71d9793b4459e8e0fda5237f5cfa0" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 44295 } }
I20260808 08:25:56.696185 8379 leader_election.cc:302] T 00000000000000000000000000000000 P 5de71d9793b4459e8e0fda5237f5cfa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5de71d9793b4459e8e0fda5237f5cfa0; no voters:
I20260808 08:25:56.696501 8379 leader_election.cc:288] T 00000000000000000000000000000000 P 5de71d9793b4459e8e0fda5237f5cfa0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:25:56.696590 8389 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5de71d9793b4459e8e0fda5237f5cfa0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:25:56.696848 8389 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5de71d9793b4459e8e0fda5237f5cfa0 [term 1 LEADER]: Becoming Leader. State: Replica: 5de71d9793b4459e8e0fda5237f5cfa0, State: Running, Role: LEADER
I20260808 08:25:56.697214 8389 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5de71d9793b4459e8e0fda5237f5cfa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5de71d9793b4459e8e0fda5237f5cfa0" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 44295 } }
I20260808 08:25:56.697760 8379 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5de71d9793b4459e8e0fda5237f5cfa0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:25:56.699115 8393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5de71d9793b4459e8e0fda5237f5cfa0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5de71d9793b4459e8e0fda5237f5cfa0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5de71d9793b4459e8e0fda5237f5cfa0" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 44295 } } }
I20260808 08:25:56.699234 8393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5de71d9793b4459e8e0fda5237f5cfa0 [sys.catalog]: This master's current role is: LEADER
I20260808 08:25:56.699251 8385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:56.685399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53304 (local address 127.22.235.254:44295)
0808 08:25:56.685795 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:56.685810 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:56.685845 (+ 35us) server_negotiation.cc:407] Connection header received
0808 08:25:56.686438 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:56.686454 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:56.686796 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:56.687080 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:56.687936 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:56.689146 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:56.690245 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:56.690780 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:56.690899 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:56.693530 (+ 2631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:56.693535 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:56.693548 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:56.693576 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:56.695325 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:56.695735 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:56.695742 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:56.695747 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:56.695862 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:56.696163 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:56.696166 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:56.696167 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:56.696654 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:56.696860 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:56.697589 (+ 729us) server_negotiation.cc:299] Negotiation successful
0808 08:25:56.697877 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":108,"threads_started":1}
I20260808 08:25:56.700377 8394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5de71d9793b4459e8e0fda5237f5cfa0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5de71d9793b4459e8e0fda5237f5cfa0. Latest consensus state: current_term: 1 leader_uuid: "5de71d9793b4459e8e0fda5237f5cfa0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5de71d9793b4459e8e0fda5237f5cfa0" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 44295 } } }
I20260808 08:25:56.700528 8394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5de71d9793b4459e8e0fda5237f5cfa0 [sys.catalog]: This master's current role is: LEADER
W20260808 08:25:56.703197 8416 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5de71d9793b4459e8e0fda5237f5cfa0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 08:25:56.703272 8416 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 08:25:56.703334 8407 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:25:56.703994 8407 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:25:56.709208 8407 catalog_manager.cc:1347] Generated new cluster ID: 7d661b933dd1405586a6c5bb7527ed8e
I20260808 08:25:56.709275 8407 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:25:56.744673 8407 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:25:56.745630 8407 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:25:56.755586 8407 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5de71d9793b4459e8e0fda5237f5cfa0: Generated new TSK 0
I20260808 08:25:56.756263 8407 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.235.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab.
Entry for principal kudu/127.22.235.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab.
I20260808 08:25:56.791687 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.193:0
--local_ip_for_outbound_sockets=127.22.235.193
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:44295
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab
--principal=kudu/127.22.235.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:40327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:25:56.906548 8423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:56.906829 8423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:56.906873 8423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:56.910923 8423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:56.910981 8423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:56.911036 8423 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.193
I20260808 08:25:56.916106 8423 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:40327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab
--principal=kudu/127.22.235.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:44295
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8423
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.193
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:56.917151 8423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:56.918188 8423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:56.927793 8434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:56.927970 8423 server_base.cc:1034] running on GCE node
W20260808 08:25:56.927794 8433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:56.927822 8437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:56.928511 8423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:56.929171 8423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:56.930353 8423 hybrid_clock.cc:648] HybridClock initialized: now 1786177556930334 us; error 28 us; skew 500 ppm
Aug 08 08:25:56 dist-test-slave-f7th krb5kdc[8222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177556, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:56.933002 8423 init.cc:377] Logged in from keytab as kudu/127.22.235.193@KRBTEST.COM (short username kudu)
I20260808 08:25:56.933856 8423 webserver.cc:460] Webserver started at http://127.22.235.193:41407/ using document root <none> and password file <none>
I20260808 08:25:56.934407 8423 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:56.934473 8423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:56.934688 8423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:56.936708 8423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/data/instance:
uuid: "5e3503f977d3439cb09b2630113526d9"
format_stamp: "Formatted at 2026-08-08 08:25:56 on dist-test-slave-f7th"
server_key: "e489ae4b621ebc8581e441b40d2310d8"
server_key_iv: "7fa45d04c489ed5cd2b4e42a2893cf08"
server_key_version: "encryptionkey@0"
I20260808 08:25:56.937243 8423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance:
uuid: "5e3503f977d3439cb09b2630113526d9"
format_stamp: "Formatted at 2026-08-08 08:25:56 on dist-test-slave-f7th"
server_key: "e489ae4b621ebc8581e441b40d2310d8"
server_key_iv: "7fa45d04c489ed5cd2b4e42a2893cf08"
server_key_version: "encryptionkey@0"
I20260808 08:25:56.940794 8423 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 08:25:56.943117 8449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:56.944252 8423 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:25:56.944370 8423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/wal
uuid: "5e3503f977d3439cb09b2630113526d9"
format_stamp: "Formatted at 2026-08-08 08:25:56 on dist-test-slave-f7th"
server_key: "e489ae4b621ebc8581e441b40d2310d8"
server_key_iv: "7fa45d04c489ed5cd2b4e42a2893cf08"
server_key_version: "encryptionkey@0"
I20260808 08:25:56.944458 8423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:56.957980 8423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:56.960590 8423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:56.960786 8423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:56.961385 8423 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:25:56.962196 8423 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:25:56.962244 8423 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:56.962296 8423 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:25:56.962324 8423 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:56.970791 8423 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.193:44559
I20260808 08:25:56.970824 8658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.193:44559 every 8 connection(s)
I20260808 08:25:56.971627 8423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
I20260808 08:25:56.978485 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 8423
I20260808 08:25:56.978610 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance
I20260808 08:25:56.978931 23471 external_mini_cluster.cc:1428] Setting key cea38461483496afabce6b9e27093af2
Aug 08 08:25:56 dist-test-slave-f7th krb5kdc[8222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177556, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:25:56.984508 8385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:56.973324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:42627 (local address 127.22.235.254:44295)
0808 08:25:56.973485 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:56.973490 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:56.974267 (+ 777us) server_negotiation.cc:407] Connection header received
0808 08:25:56.975567 (+ 1300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:56.975570 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:56.975622 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:56.975693 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:56.976850 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:56.977381 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:56.978429 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:56.978756 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:56.978861 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:56.981151 (+ 2290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:56.981155 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:56.981158 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:56.981187 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:56.982575 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:56.983125 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:56.983127 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:56.983129 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:56.983197 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:56.983536 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:56.983538 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:56.983539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:56.983695 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:56.983763 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:56.984294 (+ 531us) server_negotiation.cc:299] Negotiation successful
0808 08:25:56.984387 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 08:25:56.985656 8666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:56.973706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:44295 (local address 127.22.235.193:42627)
0808 08:25:56.974124 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:56.974165 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:56.975375 (+ 1210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:56.975846 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:56.975853 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:56.976194 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:56.976714 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:56.976730 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:56.977482 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:56.977484 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:56.978323 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:56.978332 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:56.978894 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:56.978897 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:56.979046 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:56.979583 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:56.979599 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:56.980992 (+ 1393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:56.982683 (+ 1691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:56.982690 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:56.982705 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:56.983041 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:56.983269 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:56.983271 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:56.983273 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:56.983448 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:56.983787 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:56.983793 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:56.984075 (+ 282us) client_negotiation.cc:769] Sending connection context
0808 08:25:56.984274 (+ 199us) client_negotiation.cc:240] Negotiation successful
0808 08:25:56.984598 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":103,"threads_started":1}
I20260808 08:25:56.986815 8661 heartbeater.cc:347] Connected to a master server at 127.22.235.254:44295
I20260808 08:25:56.987083 8661 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:56.987673 8661 heartbeater.cc:511] Master 127.22.235.254:44295 requested a full tablet report, sending...
I20260808 08:25:56.989259 8305 ts_manager.cc:194] Registered new tserver with Master: 5e3503f977d3439cb09b2630113526d9 (127.22.235.193:44559)
WARNING: no policy specified for kudu/127.22.235.194@KRBTEST.COM; defaulting to no policy
I20260808 08:25:56.990388 8305 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.193@KRBTEST.COM'} at 127.22.235.193:42627
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab.
Entry for principal kudu/127.22.235.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab.
I20260808 08:25:57.006959 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.194:0
--local_ip_for_outbound_sockets=127.22.235.194
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:44295
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab
--principal=kudu/127.22.235.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:40327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:25:57.117724 8673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:57.118001 8673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:57.118065 8673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:57.122043 8673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:57.122102 8673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:57.122180 8673 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.194
I20260808 08:25:57.127321 8673 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:40327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab
--principal=kudu/127.22.235.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:44295
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8673
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.194
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:57.128315 8673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:57.129339 8673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:57.138739 8682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:57.138713 8683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:57.138796 8673 server_base.cc:1034] running on GCE node
W20260808 08:25:57.138744 8686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:57.139345 8673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:57.139952 8673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:57.141106 8673 hybrid_clock.cc:648] HybridClock initialized: now 1786177557141086 us; error 31 us; skew 500 ppm
Aug 08 08:25:57 dist-test-slave-f7th krb5kdc[8222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177557, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:57.143769 8673 init.cc:377] Logged in from keytab as kudu/127.22.235.194@KRBTEST.COM (short username kudu)
I20260808 08:25:57.144603 8673 webserver.cc:460] Webserver started at http://127.22.235.194:38453/ using document root <none> and password file <none>
I20260808 08:25:57.145207 8673 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:57.145267 8673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:57.145460 8673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:57.147537 8673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/data/instance:
uuid: "c7c317f0fb1746c485ab909a992e9e81"
format_stamp: "Formatted at 2026-08-08 08:25:57 on dist-test-slave-f7th"
server_key: "bcb6186116f3ba9cae6215c00de2c69e"
server_key_iv: "4c2ab2f597a7e945de408c02e286a91a"
server_key_version: "encryptionkey@0"
I20260808 08:25:57.148114 8673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance:
uuid: "c7c317f0fb1746c485ab909a992e9e81"
format_stamp: "Formatted at 2026-08-08 08:25:57 on dist-test-slave-f7th"
server_key: "bcb6186116f3ba9cae6215c00de2c69e"
server_key_iv: "4c2ab2f597a7e945de408c02e286a91a"
server_key_version: "encryptionkey@0"
I20260808 08:25:57.151947 8673 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260808 08:25:57.154428 8697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:57.155613 8673 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 08:25:57.155714 8673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/wal
uuid: "c7c317f0fb1746c485ab909a992e9e81"
format_stamp: "Formatted at 2026-08-08 08:25:57 on dist-test-slave-f7th"
server_key: "bcb6186116f3ba9cae6215c00de2c69e"
server_key_iv: "4c2ab2f597a7e945de408c02e286a91a"
server_key_version: "encryptionkey@0"
I20260808 08:25:57.155812 8673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:57.166240 8673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:57.168879 8673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:57.169035 8673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:57.169579 8673 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:25:57.170477 8673 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:25:57.170537 8673 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:57.170576 8673 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:25:57.170596 8673 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:57.178692 8673 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.194:40933
I20260808 08:25:57.178717 8902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.194:40933 every 8 connection(s)
I20260808 08:25:57.179533 8673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
I20260808 08:25:57.183071 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 8673
I20260808 08:25:57.183187 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance
I20260808 08:25:57.183478 23471 external_mini_cluster.cc:1428] Setting key 969c324b3cd990b684483fea27c8ecb4
Aug 08 08:25:57 dist-test-slave-f7th krb5kdc[8222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177557, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.194@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.235.195@KRBTEST.COM; defaulting to no policy
I20260808 08:25:57.194460 8385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:57.181241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:44391 (local address 127.22.235.254:44295)
0808 08:25:57.181365 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:57.181369 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:57.182177 (+ 808us) server_negotiation.cc:407] Connection header received
0808 08:25:57.183726 (+ 1549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:57.183729 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:57.183779 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:57.183848 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:57.185178 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:57.185697 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:57.187255 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:57.187619 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:57.187721 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:57.190943 (+ 3222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:57.190947 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:57.190950 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:57.190988 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:57.192393 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:57.192928 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:57.192932 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:57.192934 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:57.193020 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:57.193349 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:57.193353 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:57.193355 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:57.193559 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:57.193647 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:57.194163 (+ 516us) server_negotiation.cc:299] Negotiation successful
0808 08:25:57.194294 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 08:25:57.195253 8907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:57.181591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:44295 (local address 127.22.235.194:44391)
0808 08:25:57.182032 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:57.182070 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:57.183506 (+ 1436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:57.184026 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:57.184033 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:57.184436 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:57.185032 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:57.185052 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:57.185825 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:57.185829 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:57.187120 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:57.187133 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:57.187900 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:57.187904 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:57.188108 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:57.188811 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:57.188834 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:57.190777 (+ 1943us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:57.192506 (+ 1729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:57.192512 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:57.192528 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:57.192831 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:57.193104 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:57.193107 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:57.193109 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:57.193249 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:57.193661 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:57.193668 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:57.193912 (+ 244us) client_negotiation.cc:769] Sending connection context
0808 08:25:57.194083 (+ 171us) client_negotiation.cc:240] Negotiation successful
0808 08:25:57.194404 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":115,"threads_started":1}
I20260808 08:25:57.196523 8903 heartbeater.cc:347] Connected to a master server at 127.22.235.254:44295
I20260808 08:25:57.196846 8903 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:57.197453 8903 heartbeater.cc:511] Master 127.22.235.254:44295 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.195@KRBTEST.COM" created.
I20260808 08:25:57.198520 8305 ts_manager.cc:194] Registered new tserver with Master: c7c317f0fb1746c485ab909a992e9e81 (127.22.235.194:40933)
I20260808 08:25:57.199154 8305 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.194@KRBTEST.COM'} at 127.22.235.194:44391
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab.
Entry for principal kudu/127.22.235.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab.
I20260808 08:25:57.212263 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.195:0
--local_ip_for_outbound_sockets=127.22.235.195
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:44295
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab
--principal=kudu/127.22.235.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:40327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:25:57.325515 8911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:25:57.325752 8911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:25:57.325794 8911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:25:57.329744 8911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:25:57.329795 8911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:25:57.329865 8911 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.195
I20260808 08:25:57.334934 8911 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:40327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab
--principal=kudu/127.22.235.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:44295
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.195
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:25:57.335908 8911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:25:57.336877 8911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:25:57.346211 8931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:25:57.346172 8926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:57.346279 8911 server_base.cc:1034] running on GCE node
W20260808 08:25:57.346166 8929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:25:57.346768 8911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:25:57.347389 8911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:25:57.348573 8911 hybrid_clock.cc:648] HybridClock initialized: now 1786177557348542 us; error 40 us; skew 500 ppm
Aug 08 08:25:57 dist-test-slave-f7th krb5kdc[8222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177557, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:25:57.351199 8911 init.cc:377] Logged in from keytab as kudu/127.22.235.195@KRBTEST.COM (short username kudu)
I20260808 08:25:57.352030 8911 webserver.cc:460] Webserver started at http://127.22.235.195:43543/ using document root <none> and password file <none>
I20260808 08:25:57.352571 8911 fs_manager.cc:359] Metadata directory not provided
I20260808 08:25:57.352666 8911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:25:57.352880 8911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:25:57.354858 8911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/data/instance:
uuid: "773576c2404e45999f256d997fda709f"
format_stamp: "Formatted at 2026-08-08 08:25:57 on dist-test-slave-f7th"
server_key: "c3c778caebcfa860c1e61d79339fa28c"
server_key_iv: "ae4095f29d3d9f27580d85cf0ff51760"
server_key_version: "encryptionkey@0"
I20260808 08:25:57.355372 8911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance:
uuid: "773576c2404e45999f256d997fda709f"
format_stamp: "Formatted at 2026-08-08 08:25:57 on dist-test-slave-f7th"
server_key: "c3c778caebcfa860c1e61d79339fa28c"
server_key_iv: "ae4095f29d3d9f27580d85cf0ff51760"
server_key_version: "encryptionkey@0"
I20260808 08:25:57.358964 8911 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 08:25:57.361330 8940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:57.362499 8911 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:25:57.362602 8911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/wal
uuid: "773576c2404e45999f256d997fda709f"
format_stamp: "Formatted at 2026-08-08 08:25:57 on dist-test-slave-f7th"
server_key: "c3c778caebcfa860c1e61d79339fa28c"
server_key_iv: "ae4095f29d3d9f27580d85cf0ff51760"
server_key_version: "encryptionkey@0"
I20260808 08:25:57.362689 8911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:25:57.382974 8911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:25:57.385950 8911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:25:57.386118 8911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:25:57.386698 8911 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:25:57.387537 8911 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:25:57.387584 8911 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:57.387638 8911 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:25:57.387662 8911 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:25:57.396209 8911 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.195:37873
I20260808 08:25:57.396281 9152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.195:37873 every 8 connection(s)
I20260808 08:25:57.397106 8911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
I20260808 08:25:57.398381 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 8911
I20260808 08:25:57.398496 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance
I20260808 08:25:57.398761 23471 external_mini_cluster.cc:1428] Setting key e9ed52e0c1e5824aebcc375319b588a6
Aug 08 08:25:57 dist-test-slave-f7th krb5kdc[8222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177557, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.195@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:25:57.409680 8385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:57.398935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:44429 (local address 127.22.235.254:44295)
0808 08:25:57.399073 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:57.399078 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:57.400047 (+ 969us) server_negotiation.cc:407] Connection header received
0808 08:25:57.401548 (+ 1501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:57.401550 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:57.401595 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:57.401660 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:57.402718 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:57.403223 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:57.404248 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:57.404445 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:57.404516 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:57.406708 (+ 2192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:57.406711 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:57.406713 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:57.406737 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:57.407974 (+ 1237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:57.408375 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:57.408377 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:57.408379 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:57.408445 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:57.408730 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:57.408732 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:57.408733 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:57.408902 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:57.408976 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:57.409389 (+ 413us) server_negotiation.cc:299] Negotiation successful
0808 08:25:57.409488 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 08:25:57.410408 9155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:57.399369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:44295 (local address 127.22.235.195:44429)
0808 08:25:57.399903 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:57.399953 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:57.401353 (+ 1400us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:57.401795 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:57.401801 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:57.402137 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:25:57.402611 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:57.402625 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:57.403307 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:57.403309 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:57.404157 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:57.404167 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:57.404546 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:57.404549 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:57.404704 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:57.405170 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:25:57.405185 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:25:57.406570 (+ 1385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:25:57.408058 (+ 1488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:57.408062 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:57.408072 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:57.408305 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:57.408505 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:25:57.408507 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:25:57.408508 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:25:57.408665 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:25:57.408977 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:25:57.408982 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:25:57.409201 (+ 219us) client_negotiation.cc:769] Sending connection context
0808 08:25:57.409345 (+ 144us) client_negotiation.cc:240] Negotiation successful
0808 08:25:57.409599 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":108,"threads_started":1}
I20260808 08:25:57.411463 9153 heartbeater.cc:347] Connected to a master server at 127.22.235.254:44295
I20260808 08:25:57.411697 9153 heartbeater.cc:464] Registering TS with master...
I20260808 08:25:57.412169 9153 heartbeater.cc:511] Master 127.22.235.254:44295 requested a full tablet report, sending...
I20260808 08:25:57.413117 8305 ts_manager.cc:194] Registered new tserver with Master: 773576c2404e45999f256d997fda709f (127.22.235.195:37873)
I20260808 08:25:57.413667 8305 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.195@KRBTEST.COM'} at 127.22.235.195:44429
I20260808 08:25:57.423799 23471 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 08:25:57.431901 8385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:57.425644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53318 (local address 127.22.235.254:44295)
0808 08:25:57.425765 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:57.425769 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:57.425913 (+ 144us) server_negotiation.cc:407] Connection header received
0808 08:25:57.426056 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:57.426058 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:57.426097 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:57.426150 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:25:57.426751 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:57.427244 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:57.427993 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:57.428187 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:57.428249 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:57.429426 (+ 1177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:25:57.429429 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:25:57.429431 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:25:57.429454 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:25:57.430674 (+ 1220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:57.431000 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:57.431003 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:57.431004 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:57.431068 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:25:57.431245 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:25:57.431246 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:25:57.431247 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:25:57.431436 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:25:57.431542 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:57.431673 (+ 131us) server_negotiation.cc:299] Negotiation successful
0808 08:25:57.431756 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 08:25:57.434444 8305 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53318:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 08:25:57.437354 8305 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 08:25:57.451128 9176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:57.446041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44874 (local address 127.22.235.193:44559)
0808 08:25:57.446478 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:57.446494 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:57.446516 (+ 22us) server_negotiation.cc:407] Connection header received
0808 08:25:57.446591 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:57.446596 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:57.446773 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:57.447201 (+ 428us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:25:57.447374 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:57.448523 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:57.449692 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:57.450454 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:57.450575 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:57.450707 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:57.450868 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 08:25:57.450959 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":93,"threads_started":1}
I20260808 08:25:57.451125 9171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:57.445233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.194:40933 (local address 127.0.0.1:53290)
0808 08:25:57.445562 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:57.445612 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:57.445735 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:57.446613 (+ 878us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:57.446620 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:57.446644 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:25:57.446777 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:57.446799 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:57.448450 (+ 1651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:57.448453 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:57.449769 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:57.449778 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:57.450754 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:57.450757 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:57.450836 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:57.450857 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 08:25:57.450898 (+ 41us) client_negotiation.cc:240] Negotiation successful
0808 08:25:57.450950 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":34,"thread_start_us":67,"threads_started":1}
I20260808 08:25:57.451124 9173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:57.445601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:44559 (local address 127.0.0.1:44874)
0808 08:25:57.445853 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:57.445866 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:57.445959 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:57.446933 (+ 974us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:57.446936 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:57.446955 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:25:57.447053 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:57.447060 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:57.448652 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:57.448656 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:57.449560 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:57.449571 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:57.450569 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:57.450573 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:57.450673 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:57.450721 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 08:25:57.450835 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 08:25:57.450947 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":35,"thread_start_us":97,"threads_started":1}
I20260808 08:25:57.451638 9174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:57.445948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.195:37873 (local address 127.0.0.1:52888)
0808 08:25:57.446152 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:57.446165 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:57.446243 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:57.447110 (+ 867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:57.447113 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:57.447130 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:25:57.447230 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:57.447237 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:57.448383 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:57.448386 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:57.450239 (+ 1853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:57.450249 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:57.451275 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:57.451280 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:57.451372 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:57.451396 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 08:25:57.451437 (+ 41us) client_negotiation.cc:240] Negotiation successful
0808 08:25:57.451511 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":124,"thread_start_us":92,"threads_started":1}
I20260808 08:25:57.451668 9172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:57.445409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53290 (local address 127.22.235.194:40933)
0808 08:25:57.446104 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:57.446115 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:57.446141 (+ 26us) server_negotiation.cc:407] Connection header received
0808 08:25:57.446219 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:57.446224 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:57.446428 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:57.446949 (+ 521us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:25:57.447108 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:57.448296 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:57.449893 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:57.450629 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:57.451329 (+ 700us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:57.451453 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:57.451497 (+ 44us) server_negotiation.cc:299] Negotiation successful
0808 08:25:57.451558 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":111,"threads_started":1}
I20260808 08:25:57.452209 9177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:57.446357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52888 (local address 127.22.235.195:37873)
0808 08:25:57.446650 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:57.446661 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:57.446681 (+ 20us) server_negotiation.cc:407] Connection header received
0808 08:25:57.446752 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:57.446757 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:57.446954 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:57.447372 (+ 418us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:25:57.447491 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:57.448261 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:57.450369 (+ 2108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:57.451133 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:57.451823 (+ 690us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:57.451944 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:57.451990 (+ 46us) server_negotiation.cc:299] Negotiation successful
0808 08:25:57.452068 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":129,"threads_started":1}
I20260808 08:25:57.454499 8541 tablet_service.cc:1467] Processing CreateTablet for tablet e11608d51f4b486f9c1afa5ac2c5b6d1 (DEFAULT_TABLE table=test-table [id=34a320cf77e04a829fd1228199a7c373]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:25:57.454618 8770 tablet_service.cc:1467] Processing CreateTablet for tablet e11608d51f4b486f9c1afa5ac2c5b6d1 (DEFAULT_TABLE table=test-table [id=34a320cf77e04a829fd1228199a7c373]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:25:57.454618 9041 tablet_service.cc:1467] Processing CreateTablet for tablet e11608d51f4b486f9c1afa5ac2c5b6d1 (DEFAULT_TABLE table=test-table [id=34a320cf77e04a829fd1228199a7c373]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:25:57.455452 8541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e11608d51f4b486f9c1afa5ac2c5b6d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:57.455662 8770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e11608d51f4b486f9c1afa5ac2c5b6d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:57.455665 9041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e11608d51f4b486f9c1afa5ac2c5b6d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:25:57.462203 9178 tablet_bootstrap.cc:492] T e11608d51f4b486f9c1afa5ac2c5b6d1 P 5e3503f977d3439cb09b2630113526d9: Bootstrap starting.
I20260808 08:25:57.462908 9181 tablet_bootstrap.cc:492] T e11608d51f4b486f9c1afa5ac2c5b6d1 P c7c317f0fb1746c485ab909a992e9e81: Bootstrap starting.
I20260808 08:25:57.463636 9179 tablet_bootstrap.cc:492] T e11608d51f4b486f9c1afa5ac2c5b6d1 P 773576c2404e45999f256d997fda709f: Bootstrap starting.
I20260808 08:25:57.464543 9178 tablet_bootstrap.cc:654] T e11608d51f4b486f9c1afa5ac2c5b6d1 P 5e3503f977d3439cb09b2630113526d9: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:57.465509 9181 tablet_bootstrap.cc:654] T e11608d51f4b486f9c1afa5ac2c5b6d1 P c7c317f0fb1746c485ab909a992e9e81: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:57.465620 9179 tablet_bootstrap.cc:654] T e11608d51f4b486f9c1afa5ac2c5b6d1 P 773576c2404e45999f256d997fda709f: Neither blocks nor log segments found. Creating new log.
I20260808 08:25:57.465878 9178 log.cc:824] T e11608d51f4b486f9c1afa5ac2c5b6d1 P 5e3503f977d3439cb09b2630113526d9: Log is configured to *not* fsync() on all Append() calls
I20260808 08:25:57.466307 9181 log.cc:824] T e11608d51f4b486f9c1afa5ac2c5b6d1 P c7c317f0fb1746c485ab909a992e9e81: Log is configured to *not* fsync() on all Append() calls
I20260808 08:25:57.466341 9179 log.cc:824] T e11608d51f4b486f9c1afa5ac2c5b6d1 P 773576c2404e45999f256d997fda709f: Log is configured to *not* fsync() on all Append() calls
I20260808 08:25:57.468111 9178 tablet_bootstrap.cc:492] T e11608d51f4b486f9c1afa5ac2c5b6d1 P 5e3503f977d3439cb09b2630113526d9: No bootstrap required, opened a new log
I20260808 08:25:57.468142 9179 tablet_bootstrap.cc:492] T e11608d51f4b486f9c1afa5ac2c5b6d1 P 773576c2404e45999f256d997fda709f: No bootstrap required, opened a new log
I20260808 08:25:57.468312 9179 ts_tablet_manager.cc:1397] T e11608d51f4b486f9c1afa5ac2c5b6d1 P 773576c2404e45999f256d997fda709f: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 08:25:57.468313 9178 ts_tablet_manager.cc:1397] T e11608d51f4b486f9c1afa5ac2c5b6d1 P 5e3503f977d3439cb09b2630113526d9: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 08:25:57.468423 9181 tablet_bootstrap.cc:492] T e11608d51f4b486f9c1afa5ac2c5b6d1 P c7c317f0fb1746c485ab909a992e9e81: No bootstrap required, opened a new log
I20260808 08:25:57.468597 9181 ts_tablet_manager.cc:1397] T e11608d51f4b486f9c1afa5ac2c5b6d1 P c7c317f0fb1746c485ab909a992e9e81: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 08:25:57.471244 9181 raft_consensus.cc:348] T e11608d51f4b486f9c1afa5ac2c5b6d1 P c7c317f0fb1746c485ab909a992e9e81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7c317f0fb1746c485ab909a992e9e81" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 40933 } } peers { permanent_uuid: "773576c2404e45999f256d997fda709f" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37873 } } peers { permanent_uuid: "5e3503f977d3439cb09b2630113526d9" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44559 } }
I20260808 08:25:57.471446 9181 raft_consensus.cc:374] T e11608d51f4b486f9c1afa5ac2c5b6d1 P c7c317f0fb1746c485ab909a992e9e81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:57.471517 9181 raft_consensus.cc:729] T e11608d51f4b486f9c1afa5ac2c5b6d1 P c7c317f0fb1746c485ab909a992e9e81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7c317f0fb1746c485ab909a992e9e81, State: Initialized, Role: FOLLOWER
I20260808 08:25:57.471923 9181 consensus_queue.cc:260] T e11608d51f4b486f9c1afa5ac2c5b6d1 P c7c317f0fb1746c485ab909a992e9e81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7c317f0fb1746c485ab909a992e9e81" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 40933 } } peers { permanent_uuid: "773576c2404e45999f256d997fda709f" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37873 } } peers { permanent_uuid: "5e3503f977d3439cb09b2630113526d9" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44559 } }
I20260808 08:25:57.472066 9179 raft_consensus.cc:348] T e11608d51f4b486f9c1afa5ac2c5b6d1 P 773576c2404e45999f256d997fda709f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7c317f0fb1746c485ab909a992e9e81" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 40933 } } peers { permanent_uuid: "773576c2404e45999f256d997fda709f" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37873 } } peers { permanent_uuid: "5e3503f977d3439cb09b2630113526d9" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44559 } }
I20260808 08:25:57.472067 9178 raft_consensus.cc:348] T e11608d51f4b486f9c1afa5ac2c5b6d1 P 5e3503f977d3439cb09b2630113526d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7c317f0fb1746c485ab909a992e9e81" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 40933 } } peers { permanent_uuid: "773576c2404e45999f256d997fda709f" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37873 } } peers { permanent_uuid: "5e3503f977d3439cb09b2630113526d9" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44559 } }
I20260808 08:25:57.472303 9178 raft_consensus.cc:374] T e11608d51f4b486f9c1afa5ac2c5b6d1 P 5e3503f977d3439cb09b2630113526d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:57.472343 9179 raft_consensus.cc:374] T e11608d51f4b486f9c1afa5ac2c5b6d1 P 773576c2404e45999f256d997fda709f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:25:57.472420 9179 raft_consensus.cc:729] T e11608d51f4b486f9c1afa5ac2c5b6d1 P 773576c2404e45999f256d997fda709f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 773576c2404e45999f256d997fda709f, State: Initialized, Role: FOLLOWER
I20260808 08:25:57.472443 9178 raft_consensus.cc:729] T e11608d51f4b486f9c1afa5ac2c5b6d1 P 5e3503f977d3439cb09b2630113526d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e3503f977d3439cb09b2630113526d9, State: Initialized, Role: FOLLOWER
I20260808 08:25:57.472635 8903 heartbeater.cc:503] Master 127.22.235.254:44295 was elected leader, sending a full tablet report...
I20260808 08:25:57.472892 9181 ts_tablet_manager.cc:1428] T e11608d51f4b486f9c1afa5ac2c5b6d1 P c7c317f0fb1746c485ab909a992e9e81: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 08:25:57.472910 9178 consensus_queue.cc:260] T e11608d51f4b486f9c1afa5ac2c5b6d1 P 5e3503f977d3439cb09b2630113526d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7c317f0fb1746c485ab909a992e9e81" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 40933 } } peers { permanent_uuid: "773576c2404e45999f256d997fda709f" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37873 } } peers { permanent_uuid: "5e3503f977d3439cb09b2630113526d9" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44559 } }
I20260808 08:25:57.472910 9179 consensus_queue.cc:260] T e11608d51f4b486f9c1afa5ac2c5b6d1 P 773576c2404e45999f256d997fda709f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7c317f0fb1746c485ab909a992e9e81" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 40933 } } peers { permanent_uuid: "773576c2404e45999f256d997fda709f" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37873 } } peers { permanent_uuid: "5e3503f977d3439cb09b2630113526d9" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44559 } }
I20260808 08:25:57.473606 9153 heartbeater.cc:503] Master 127.22.235.254:44295 was elected leader, sending a full tablet report...
I20260808 08:25:57.473820 9178 ts_tablet_manager.cc:1428] T e11608d51f4b486f9c1afa5ac2c5b6d1 P 5e3503f977d3439cb09b2630113526d9: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 08:25:57.473820 9179 ts_tablet_manager.cc:1428] T e11608d51f4b486f9c1afa5ac2c5b6d1 P 773576c2404e45999f256d997fda709f: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 08:25:57.474120 8661 heartbeater.cc:503] Master 127.22.235.254:44295 was elected leader, sending a full tablet report...
W20260808 08:25:57.648077 9154 tablet.cc:2369] T e11608d51f4b486f9c1afa5ac2c5b6d1 P 773576c2404e45999f256d997fda709f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 08:25:57.680639 8904 tablet.cc:2369] T e11608d51f4b486f9c1afa5ac2c5b6d1 P c7c317f0fb1746c485ab909a992e9e81: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 08:25:57.723546 8665 tablet.cc:2369] T e11608d51f4b486f9c1afa5ac2c5b6d1 P 5e3503f977d3439cb09b2630113526d9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 08:25:57.770931 9189 raft_consensus.cc:482] T e11608d51f4b486f9c1afa5ac2c5b6d1 P c7c317f0fb1746c485ab909a992e9e81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 08:25:57.771185 9189 raft_consensus.cc:504] T e11608d51f4b486f9c1afa5ac2c5b6d1 P c7c317f0fb1746c485ab909a992e9e81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7c317f0fb1746c485ab909a992e9e81" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 40933 } } peers { permanent_uuid: "773576c2404e45999f256d997fda709f" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37873 } } peers { permanent_uuid: "5e3503f977d3439cb09b2630113526d9" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44559 } }
I20260808 08:25:57.772387 9189 leader_election.cc:288] T e11608d51f4b486f9c1afa5ac2c5b6d1 P c7c317f0fb1746c485ab909a992e9e81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 773576c2404e45999f256d997fda709f (127.22.235.195:37873), 5e3503f977d3439cb09b2630113526d9 (127.22.235.193:44559)
I20260808 08:25:57.776175 9176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:57.772823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:47173 (local address 127.22.235.193:44559)
0808 08:25:57.773009 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:57.773013 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:57.773186 (+ 173us) server_negotiation.cc:407] Connection header received
0808 08:25:57.773303 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:57.773305 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:57.773361 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:57.773431 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:25:57.773706 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:57.774199 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:57.775254 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:57.775634 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:57.775737 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:57.775791 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:57.775987 (+ 196us) server_negotiation.cc:299] Negotiation successful
0808 08:25:57.776051 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260808 08:25:57.776175 9177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:57.772790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:58257 (local address 127.22.235.195:37873)
0808 08:25:57.772928 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:57.772933 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:57.773108 (+ 175us) server_negotiation.cc:407] Connection header received
0808 08:25:57.773234 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:57.773237 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:57.773292 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:57.773353 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:25:57.773622 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:57.774139 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:57.775257 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:57.775636 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:57.775743 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:57.775798 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:57.775978 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 08:25:57.776046 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 08:25:57.776190 9193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:57.772665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.195:37873 (local address 127.22.235.194:58257)
0808 08:25:57.773009 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:57.773023 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:57.773134 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:57.773374 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:57.773377 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:57.773394 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:25:57.773523 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:57.773530 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:57.774244 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:57.774247 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:57.775136 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:57.775146 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:57.775741 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:57.775745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:57.775863 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:57.775882 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 08:25:57.775940 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 08:25:57.776007 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"mutex_wait_us":31,"thread_start_us":105,"threads_started":1}
I20260808 08:25:57.776194 9194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:57.772730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:44559 (local address 127.22.235.194:47173)
0808 08:25:57.773083 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:57.773097 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 08:25:57.773214 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:25:57.773458 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:25:57.773461 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:25:57.773482 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:25:57.773601 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:57.773607 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:57.774308 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:57.774310 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:57.775137 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:25:57.775146 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:25:57.775743 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:25:57.775747 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:25:57.775876 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:57.775894 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 08:25:57.775963 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 08:25:57.776034 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":30,"thread_start_us":80,"threads_started":1}
I20260808 08:25:57.776870 9073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e11608d51f4b486f9c1afa5ac2c5b6d1" candidate_uuid: "c7c317f0fb1746c485ab909a992e9e81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "773576c2404e45999f256d997fda709f" is_pre_election: true
I20260808 08:25:57.776877 8576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e11608d51f4b486f9c1afa5ac2c5b6d1" candidate_uuid: "c7c317f0fb1746c485ab909a992e9e81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e3503f977d3439cb09b2630113526d9" is_pre_election: true
I20260808 08:25:57.777150 8576 raft_consensus.cc:2454] T e11608d51f4b486f9c1afa5ac2c5b6d1 P 5e3503f977d3439cb09b2630113526d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7c317f0fb1746c485ab909a992e9e81 in term 0.
I20260808 08:25:57.777155 9073 raft_consensus.cc:2454] T e11608d51f4b486f9c1afa5ac2c5b6d1 P 773576c2404e45999f256d997fda709f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7c317f0fb1746c485ab909a992e9e81 in term 0.
I20260808 08:25:57.777591 8699 leader_election.cc:302] T e11608d51f4b486f9c1afa5ac2c5b6d1 P c7c317f0fb1746c485ab909a992e9e81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e3503f977d3439cb09b2630113526d9, c7c317f0fb1746c485ab909a992e9e81; no voters:
I20260808 08:25:57.777872 9189 raft_consensus.cc:2781] T e11608d51f4b486f9c1afa5ac2c5b6d1 P c7c317f0fb1746c485ab909a992e9e81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 08:25:57.777952 9189 raft_consensus.cc:482] T e11608d51f4b486f9c1afa5ac2c5b6d1 P c7c317f0fb1746c485ab909a992e9e81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 08:25:57.777984 9189 raft_consensus.cc:3037] T e11608d51f4b486f9c1afa5ac2c5b6d1 P c7c317f0fb1746c485ab909a992e9e81 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:57.779007 9189 raft_consensus.cc:504] T e11608d51f4b486f9c1afa5ac2c5b6d1 P c7c317f0fb1746c485ab909a992e9e81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7c317f0fb1746c485ab909a992e9e81" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 40933 } } peers { permanent_uuid: "773576c2404e45999f256d997fda709f" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37873 } } peers { permanent_uuid: "5e3503f977d3439cb09b2630113526d9" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44559 } }
I20260808 08:25:57.779330 9189 leader_election.cc:288] T e11608d51f4b486f9c1afa5ac2c5b6d1 P c7c317f0fb1746c485ab909a992e9e81 [CANDIDATE]: Term 1 election: Requested vote from peers 773576c2404e45999f256d997fda709f (127.22.235.195:37873), 5e3503f977d3439cb09b2630113526d9 (127.22.235.193:44559)
I20260808 08:25:57.779691 9073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e11608d51f4b486f9c1afa5ac2c5b6d1" candidate_uuid: "c7c317f0fb1746c485ab909a992e9e81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "773576c2404e45999f256d997fda709f"
I20260808 08:25:57.779701 8576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e11608d51f4b486f9c1afa5ac2c5b6d1" candidate_uuid: "c7c317f0fb1746c485ab909a992e9e81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e3503f977d3439cb09b2630113526d9"
I20260808 08:25:57.779796 8576 raft_consensus.cc:3037] T e11608d51f4b486f9c1afa5ac2c5b6d1 P 5e3503f977d3439cb09b2630113526d9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:57.779804 9073 raft_consensus.cc:3037] T e11608d51f4b486f9c1afa5ac2c5b6d1 P 773576c2404e45999f256d997fda709f [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:25:57.780771 8576 raft_consensus.cc:2454] T e11608d51f4b486f9c1afa5ac2c5b6d1 P 5e3503f977d3439cb09b2630113526d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7c317f0fb1746c485ab909a992e9e81 in term 1.
I20260808 08:25:57.780948 9073 raft_consensus.cc:2454] T e11608d51f4b486f9c1afa5ac2c5b6d1 P 773576c2404e45999f256d997fda709f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7c317f0fb1746c485ab909a992e9e81 in term 1.
I20260808 08:25:57.781037 8699 leader_election.cc:302] T e11608d51f4b486f9c1afa5ac2c5b6d1 P c7c317f0fb1746c485ab909a992e9e81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e3503f977d3439cb09b2630113526d9, c7c317f0fb1746c485ab909a992e9e81; no voters:
I20260808 08:25:57.781209 9189 raft_consensus.cc:2781] T e11608d51f4b486f9c1afa5ac2c5b6d1 P c7c317f0fb1746c485ab909a992e9e81 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:25:57.781432 9189 raft_consensus.cc:686] T e11608d51f4b486f9c1afa5ac2c5b6d1 P c7c317f0fb1746c485ab909a992e9e81 [term 1 LEADER]: Becoming Leader. State: Replica: c7c317f0fb1746c485ab909a992e9e81, State: Running, Role: LEADER
I20260808 08:25:57.781872 9189 consensus_queue.cc:237] T e11608d51f4b486f9c1afa5ac2c5b6d1 P c7c317f0fb1746c485ab909a992e9e81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7c317f0fb1746c485ab909a992e9e81" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 40933 } } peers { permanent_uuid: "773576c2404e45999f256d997fda709f" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37873 } } peers { permanent_uuid: "5e3503f977d3439cb09b2630113526d9" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44559 } }
I20260808 08:25:57.785312 8305 catalog_manager.cc:5495] T e11608d51f4b486f9c1afa5ac2c5b6d1 P c7c317f0fb1746c485ab909a992e9e81 reported cstate change: term changed from 0 to 1, leader changed from <none> to c7c317f0fb1746c485ab909a992e9e81 (127.22.235.194). New cstate: current_term: 1 leader_uuid: "c7c317f0fb1746c485ab909a992e9e81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7c317f0fb1746c485ab909a992e9e81" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 40933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "773576c2404e45999f256d997fda709f" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e3503f977d3439cb09b2630113526d9" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44559 } health_report { overall_health: UNKNOWN } } }
I20260808 08:25:57.812810 9172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:57.809417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53294 (local address 127.22.235.194:40933)
0808 08:25:57.809533 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 08:25:57.809536 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:25:57.809550 (+ 14us) server_negotiation.cc:407] Connection header received
0808 08:25:57.809675 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:25:57.809677 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:25:57.809737 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:25:57.809826 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:25:57.810119 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:57.810619 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:57.811423 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:25:57.811639 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:25:57.811716 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:25:57.812018 (+ 302us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:25:57.812443 (+ 425us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 08:25:57.812544 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 08:25:57.812574 (+ 30us) server_negotiation.cc:299] Negotiation successful
0808 08:25:57.812664 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 08:25:57.818552 8576 raft_consensus.cc:1261] T e11608d51f4b486f9c1afa5ac2c5b6d1 P 5e3503f977d3439cb09b2630113526d9 [term 1 FOLLOWER]: Refusing update from remote peer c7c317f0fb1746c485ab909a992e9e81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:25:57.818552 9073 raft_consensus.cc:1261] T e11608d51f4b486f9c1afa5ac2c5b6d1 P 773576c2404e45999f256d997fda709f [term 1 FOLLOWER]: Refusing update from remote peer c7c317f0fb1746c485ab909a992e9e81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:25:57.819336 9189 consensus_queue.cc:1035] T e11608d51f4b486f9c1afa5ac2c5b6d1 P c7c317f0fb1746c485ab909a992e9e81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e3503f977d3439cb09b2630113526d9" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:25:57.819530 9195 consensus_queue.cc:1035] T e11608d51f4b486f9c1afa5ac2c5b6d1 P c7c317f0fb1746c485ab909a992e9e81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "773576c2404e45999f256d997fda709f" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:25:57.838377 8303 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:25:57.838594 8303 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:25:57.841210 8303 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5de71d9793b4459e8e0fda5237f5cfa0: Sending DeleteTablet for 3 replicas of tablet e11608d51f4b486f9c1afa5ac2c5b6d1
I20260808 08:25:57.842191 9041 tablet_service.cc:1514] Processing DeleteTablet for tablet e11608d51f4b486f9c1afa5ac2c5b6d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:25:57 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:52888
I20260808 08:25:57.842226 8541 tablet_service.cc:1514] Processing DeleteTablet for tablet e11608d51f4b486f9c1afa5ac2c5b6d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:25:57 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:44874
I20260808 08:25:57.842797 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 8423
I20260808 08:25:57.843274 8770 tablet_service.cc:1514] Processing DeleteTablet for tablet e11608d51f4b486f9c1afa5ac2c5b6d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:25:57 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:53290
I20260808 08:25:57.845896 9226 tablet_replica.cc:330] T e11608d51f4b486f9c1afa5ac2c5b6d1 P c7c317f0fb1746c485ab909a992e9e81: stopping tablet replica
I20260808 08:25:57.846333 9226 raft_consensus.cc:2229] T e11608d51f4b486f9c1afa5ac2c5b6d1 P c7c317f0fb1746c485ab909a992e9e81 [term 1 LEADER]: Raft consensus shutting down.
I20260808 08:25:57.846881 9226 raft_consensus.cc:2258] T e11608d51f4b486f9c1afa5ac2c5b6d1 P c7c317f0fb1746c485ab909a992e9e81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:25:57.848052 9219 tablet_replica.cc:330] T e11608d51f4b486f9c1afa5ac2c5b6d1 P 773576c2404e45999f256d997fda709f: stopping tablet replica
I20260808 08:25:57.848393 9219 raft_consensus.cc:2229] T e11608d51f4b486f9c1afa5ac2c5b6d1 P 773576c2404e45999f256d997fda709f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 08:25:57.848981 9219 raft_consensus.cc:2258] T e11608d51f4b486f9c1afa5ac2c5b6d1 P 773576c2404e45999f256d997fda709f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:25:57.852813 9226 ts_tablet_manager.cc:1905] T e11608d51f4b486f9c1afa5ac2c5b6d1 P c7c317f0fb1746c485ab909a992e9e81: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:25:57.853861 9219 ts_tablet_manager.cc:1905] T e11608d51f4b486f9c1afa5ac2c5b6d1 P 773576c2404e45999f256d997fda709f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:25:57.856256 9226 ts_tablet_manager.cc:1918] T e11608d51f4b486f9c1afa5ac2c5b6d1 P c7c317f0fb1746c485ab909a992e9e81: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 08:25:57.856395 9226 log.cc:1196] T e11608d51f4b486f9c1afa5ac2c5b6d1 P c7c317f0fb1746c485ab909a992e9e81: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-1/wal/wals/e11608d51f4b486f9c1afa5ac2c5b6d1
I20260808 08:25:57.856662 9219 ts_tablet_manager.cc:1918] T e11608d51f4b486f9c1afa5ac2c5b6d1 P 773576c2404e45999f256d997fda709f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 08:25:57.856760 9219 log.cc:1196] T e11608d51f4b486f9c1afa5ac2c5b6d1 P 773576c2404e45999f256d997fda709f: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786177493204855-23471-0/minicluster-data/ts-2/wal/wals/e11608d51f4b486f9c1afa5ac2c5b6d1
I20260808 08:25:57.856976 9226 ts_tablet_manager.cc:1939] T e11608d51f4b486f9c1afa5ac2c5b6d1 P c7c317f0fb1746c485ab909a992e9e81: Deleting consensus metadata
I20260808 08:25:57.857208 9219 ts_tablet_manager.cc:1939] T e11608d51f4b486f9c1afa5ac2c5b6d1 P 773576c2404e45999f256d997fda709f: Deleting consensus metadata
I20260808 08:25:57.861868 8283 catalog_manager.cc:4841] TS 773576c2404e45999f256d997fda709f (127.22.235.195:37873): tablet e11608d51f4b486f9c1afa5ac2c5b6d1 (table test-table [id=34a320cf77e04a829fd1228199a7c373]) successfully deleted
I20260808 08:25:57.862105 8283 catalog_manager.cc:4841] TS c7c317f0fb1746c485ab909a992e9e81 (127.22.235.194:40933): tablet e11608d51f4b486f9c1afa5ac2c5b6d1 (table test-table [id=34a320cf77e04a829fd1228199a7c373]) successfully deleted
W20260808 08:25:57.871133 8282 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.235.193:44559 (error 108)
I20260808 08:25:57.871237 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 8673
I20260808 08:25:57.873998 9174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:25:57.873679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:44559 (local address 127.0.0.1:44880)
0808 08:25:57.873853 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 08:25:57.873913 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.193:44559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
W20260808 08:25:57.874161 8282 catalog_manager.cc:4584] TS 5e3503f977d3439cb09b2630113526d9 (127.22.235.193:44559): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e11608d51f4b486f9c1afa5ac2c5b6d1: Network error: Client connection negotiation failed: client connection to 127.22.235.193:44559: connect: Connection refused (error 111)
I20260808 08:25:57.886839 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 8911
I20260808 08:25:57.898386 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 8250
2026-08-08T08:25:57Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2681 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 08:25:57 dist-test-slave-f7th krb5kdc[9233](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 08:25:57 dist-test-slave-f7th krb5kdc[9233](info): set up 2 sockets
Aug 08 08:25:57 dist-test-slave-f7th krb5kdc[9233](info): commencing operation
krb5kdc: starting...
W20260808 08:25:59.077033 23471 mini_kdc.cc:121] Time spent starting KDC: real 1.150s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Aug 08 08:25:59 dist-test-slave-f7th krb5kdc[9233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177559, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T08:25:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:25:59Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.22.235.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.22.235.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.22.235.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/krb5kdc/rangeradmin_127.22.235.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.22.235.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/krb5kdc/rangeradmin_127.22.235.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.22.235.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.22.235.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.22.235.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/krb5kdc/rangerlookup_127.22.235.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.22.235.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/krb5kdc/rangerlookup_127.22.235.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.22.235.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.22.235.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.22.235.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/krb5kdc/HTTP_127.22.235.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.22.235.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/krb5kdc/HTTP_127.22.235.212@KRBTEST.COM.keytab.
I20260808 08:25:59.240875 23471 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskGDnrCb/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/postgres -l logfile start
2026-08-08 08:26:01.535 UTC [9282] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 08:26:01.535 UTC [9282] LOG: listening on IPv4 address "127.22.235.212", port 44809
2026-08-08 08:26:01.538 UTC [9282] LOG: listening on Unix socket "/tmp/.s.PGSQL.44809"
2026-08-08 08:26:01.558 UTC [9290] LOG: database system was shut down at 2026-08-08 08:26:00 UTC
2026-08-08 08:26:01.561 UTC [9282] LOG: database system is ready to accept connections
I20260808 08:26:02.631824 23471 mini_postgres.cc:96] Postgres bound to 44809
2026-08-08 08:26:02.640 UTC [9300] FATAL: database "slave" does not exist
127.22.235.212:44809 - accepting connections
I20260808 08:26:02.641580 23471 mini_ranger.cc:161] Starting Ranger...
I20260808 08:26:02.657572 23471 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 08:26:03.068696 23471 mini_ranger.cc:87] Created ranger Postgres database
I20260808 08:26:03.068791 23471 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ranger-admin
2026-08-08 08:26:03,370 [I] DB FLAVOR :POSTGRES
2026-08-08 08:26:03,371 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 08:26:03,371 [I] Checking connection..
2026-08-08 08:26:03,371 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.235.212:44809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 08:26:03,618 [I] Checking connection passed.
2026-08-08 08:26:03,619 [I] --------- Verifying version history table ---------
2026-08-08 08:26:03,619 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.235.212:44809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 08:26:03,874 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 08:26:03,875 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.235.212:44809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 08:26:04,115 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 08:26:04,115 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 08:26:04,115 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.235.212:44809/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 08:26:04.350 UTC [9405] WARNING: there is no transaction in progress
2026-08-08 08:26:04,356 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 08:26:04,356 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.235.212:44809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 08:26:04,612 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 08:26:04,612 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 08:26:04,612 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.235.212:44809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 08:26:04,868 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.235.212:44809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 08:26:05,116 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.235.212:44809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-f7th.c.gcp-upstream.internal','N') ;"
2026-08-08 08:26:05,355 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.235.212:44809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 08:26:05,612 [I] Table x_portal_user does not exist in database ranger
2026-08-08 08:26:05,612 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.235.212:44809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 08:26:05,860 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 08:26:05,860 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.235.212:44809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 08:26:06,102 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.235.212:44809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 08:26:06,343 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 08:26:06,343 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.235.212:44809/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 08:26:07.224 UTC [9588] WARNING: there is no transaction in progress
2026-08-08 08:26:07.239 UTC [9588] WARNING: there is no transaction in progress
2026-08-08 08:26:07.254 UTC [9588] WARNING: there is no transaction in progress
2026-08-08 08:26:07.268 UTC [9588] WARNING: there is no transaction in progress
2026-08-08 08:26:07.283 UTC [9588] WARNING: there is no transaction in progress
2026-08-08 08:26:07.298 UTC [9588] WARNING: there is no transaction in progress
2026-08-08 08:26:07.349 UTC [9588] WARNING: there is no transaction in progress
2026-08-08 08:26:07.355 UTC [9588] WARNING: there is no transaction in progress
2026-08-08 08:26:07.368 UTC [9588] WARNING: there is no transaction in progress
2026-08-08 08:26:07.378 UTC [9588] WARNING: there is no transaction in progress
2026-08-08 08:26:07.387 UTC [9588] WARNING: there is no transaction in progress
2026-08-08 08:26:07.401 UTC [9588] WARNING: there is no transaction in progress
2026-08-08 08:26:07.409 UTC [9588] WARNING: there is no transaction in progress
2026-08-08 08:26:07.415 UTC [9588] WARNING: there is no transaction in progress
2026-08-08 08:26:07.421 UTC [9588] WARNING: there is no transaction in progress
2026-08-08 08:26:07.427 UTC [9588] WARNING: there is no transaction in progress
2026-08-08 08:26:07,967 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 08:26:07,967 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.235.212:44809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 08:26:08,219 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 08:26:08,220 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.235.212:44809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 08:26:08,480 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 08:26:08,480 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.235.212:44809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 08:26:08,728 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 08:26:08,728 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.235.212:44809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 08:26:08,991 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.235.212:44809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 08:26:09,254 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.235.212:44809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-f7th.c.gcp-upstream.internal';"
2026-08-08 08:26:09,504 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 08:26:09,504 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.235.212:44809/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 08:26:09,748 [I] DB_PATCHES have already been applied
I20260808 08:26:09.753482 23471 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ranger-admin:/tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskGDnrCb/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 08:26:09.753548 23471 mini_ranger.cc:193] Using host: 127.22.235.212
I20260808 08:26:09.757862 23471 mini_ranger.cc:227] Ranger admin URL: http://127.22.235.212:50241
Aug 08, 2026 8:26:10 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 8:26:10 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 8:26:10 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 8:26:10 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 8:26:10 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 8:26:10 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 8:26:10 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 08:26:11.027083 23471 mini_ranger.cc:160] Time spent starting Ranger: real 8.386s user 0.008s sys 0.004s
Aug 08, 2026 8:26:30 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 8:26:30 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 08, 2026 8:26:30 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 8:26:30 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 8:26:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 8:26:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 8:26:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 8:26:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 8:26:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 8:26:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 8:26:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 8:26:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 8:26:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 8:26:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 8:26:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 8:26:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 8:26:30 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 8:26:31 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 08:26:31.700981 23471 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.22.235.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.22.235.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.22.235.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/krb5kdc/rangerkms_127.22.235.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.22.235.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/krb5kdc/rangerkms_127.22.235.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.22.235.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.22.235.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.22.235.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/krb5kdc/HTTP_127.22.235.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.22.235.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/krb5kdc/HTTP_127.22.235.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 08:26:31 dist-test-slave-f7th krb5kdc[9233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177591, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 08:26:31.783664 23471 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 08:26:31.793818 23471 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 08:26:32.501888 23471 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 08:26:32.501973 23471 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ranger-kms
I20260808 08:26:32.502077 23471 mini_ranger_kms.cc:227] Using postgres at 127.22.235.212:44809
2026-08-08 08:26:32,936 [I] DB FLAVOR :POSTGRES
2026-08-08 08:26:32,936 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 08:26:32,936 [I] Checking connection
2026-08-08 08:26:32,936 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.235.212:44809/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 08:26:33,226 [I] connection success
2026-08-08 08:26:33,226 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 08:26:33,226 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 08:26:33,226 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.235.212:44809/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 08:26:33,470 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 08:26:33,470 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 08:26:33,470 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 08:26:33,470 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.235.212:44809/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 08:26:33,718 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 08:26:34.168824 23471 mini_ranger_kms.cc:310] Created kms service
I20260808 08:26:34.364490 23471 mini_ranger_kms.cc:326] Created kudu user
I20260808 08:26:34.406376 23471 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 08:26:34 dist-test-slave-f7th krb5kdc[9233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177591, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.22.235.212@KRBTEST.COM
I20260808 08:26:34.785308 23471 mini_ranger_kms.cc:384] Added ranger policy
I20260808 08:26:34.785668 23471 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ranger-kms:/tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 08:26:34.785712 23471 mini_ranger_kms.cc:240] Using host: 127.22.235.212
I20260808 08:26:34.790582 23471 mini_ranger_kms.cc:276] Ranger KMS PID: 9907
I20260808 08:26:34.790716 23471 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.22.235.212:38435
Aug 08, 2026 8:26:35 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 8:26:35 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 8:26:35 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 8:26:35 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 8:26:36 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 8:26:36 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 08:26:37 dist-test-slave-f7th krb5kdc[9233](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786177597, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.235.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:26:37.279975 23471 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.496s user 0.014s sys 0.267s
I20260808 08:26:37.280097 23471 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 08:26:37.280212 23471 mini_ranger_kms.cc:401] 127.22.235.212:38435/kms/v1/keys
Aug 08 08:26:37 dist-test-slave-f7th krb5kdc[9233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177597, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.235.212@KRBTEST.COM for HTTP/127.22.235.212@KRBTEST.COM
Aug 08 08:26:37 dist-test-slave-f7th krb5kdc[9233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177597, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.235.212@KRBTEST.COM for HTTP/127.22.235.212@KRBTEST.COM
Aug 08 08:26:40 dist-test-slave-f7th krb5kdc[9233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.22.235.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
I20260808 08:26:40.321542 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:41273
--webserver_interface=127.22.235.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:45883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:41273
--ranger_config_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.235.212:38435/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:26:40.450831 10000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:26:40.451104 10000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:26:40.451161 10000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:26:40.455164 10000 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:26:40.455231 10000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:26:40.455260 10000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:26:40.455282 10000 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:26:40.455312 10000 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:26:40.461063 10000 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:45883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/master-0/wal
--ranger_kms_url=127.22.235.212:38435/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:41273
--ranger_config_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:41273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.235.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10000
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:26:40.462136 10000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:26:40.463164 10000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 08:26:40.473100 10000 server_base.cc:1034] running on GCE node
W20260808 08:26:40.473215 10008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:26:40.473299 10010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:26:40.473465 10007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:26:40.474081 10000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:26:40.475186 10000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:26:40.476368 10000 hybrid_clock.cc:648] HybridClock initialized: now 1786177600476350 us; error 29 us; skew 500 ppm
Aug 08 08:26:40 dist-test-slave-f7th krb5kdc[9233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177600, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:26:40.479547 10000 init.cc:377] Logged in from keytab as kudu/127.22.235.254@KRBTEST.COM (short username kudu)
I20260808 08:26:40.480682 10000 webserver.cc:460] Webserver started at http://127.22.235.254:35021/ using document root <none> and password file <none>
I20260808 08:26:40.481361 10000 fs_manager.cc:359] Metadata directory not provided
I20260808 08:26:40.481444 10000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:26:40.481680 10000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 08:26:40 dist-test-slave-f7th krb5kdc[9233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177600, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for HTTP/127.22.235.212@KRBTEST.COM
I20260808 08:26:40.529820 10000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "72288a3441fc49a0be342610eb2834e7"
format_stamp: "Formatted at 2026-08-08 08:26:40 on dist-test-slave-f7th"
server_key: "9dd65e303696a7510ba60b7758555da3"
server_key_iv: "8ba2e1bfbab5f4880cc4954a31803cfd"
server_key_version: "kuduclusterkey@0"
I20260808 08:26:40.530432 10000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "72288a3441fc49a0be342610eb2834e7"
format_stamp: "Formatted at 2026-08-08 08:26:40 on dist-test-slave-f7th"
server_key: "9dd65e303696a7510ba60b7758555da3"
server_key_iv: "8ba2e1bfbab5f4880cc4954a31803cfd"
server_key_version: "kuduclusterkey@0"
I20260808 08:26:40.534505 10000 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260808 08:26:40.552785 10019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:26:40.554167 10000 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 08:26:40.554306 10000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "72288a3441fc49a0be342610eb2834e7"
format_stamp: "Formatted at 2026-08-08 08:26:40 on dist-test-slave-f7th"
server_key: "9dd65e303696a7510ba60b7758555da3"
server_key_iv: "8ba2e1bfbab5f4880cc4954a31803cfd"
server_key_version: "kuduclusterkey@0"
I20260808 08:26:40.554436 10000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:26:40.566722 10000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:26:40.569618 10000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:26:40.569802 10000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:26:40.580760 10000 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.254:41273
I20260808 08:26:40.580847 10111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.254:41273 every 8 connection(s)
I20260808 08:26:40.581815 10000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:26:40.585644 10112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:26:40.589859 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 10000
I20260808 08:26:40.589972 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:26:40.590235 23471 external_mini_cluster.cc:1428] Setting key b7fc741a1cbc8d7b218c215d727f7789
I20260808 08:26:40.593247 10112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72288a3441fc49a0be342610eb2834e7: Bootstrap starting.
I20260808 08:26:40.596222 10112 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72288a3441fc49a0be342610eb2834e7: Neither blocks nor log segments found. Creating new log.
I20260808 08:26:40.597237 10112 log.cc:824] T 00000000000000000000000000000000 P 72288a3441fc49a0be342610eb2834e7: Log is configured to *not* fsync() on all Append() calls
Aug 08 08:26:40 dist-test-slave-f7th krb5kdc[9233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:26:40.599674 10112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72288a3441fc49a0be342610eb2834e7: No bootstrap required, opened a new log
I20260808 08:26:40.603585 10112 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72288a3441fc49a0be342610eb2834e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72288a3441fc49a0be342610eb2834e7" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 41273 } }
I20260808 08:26:40.603816 10112 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72288a3441fc49a0be342610eb2834e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:26:40.603899 10112 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72288a3441fc49a0be342610eb2834e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72288a3441fc49a0be342610eb2834e7, State: Initialized, Role: FOLLOWER
I20260808 08:26:40.604370 10112 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72288a3441fc49a0be342610eb2834e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72288a3441fc49a0be342610eb2834e7" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 41273 } }
I20260808 08:26:40.604501 10112 raft_consensus.cc:388] T 00000000000000000000000000000000 P 72288a3441fc49a0be342610eb2834e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:26:40.604583 10112 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72288a3441fc49a0be342610eb2834e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:26:40.604749 10112 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72288a3441fc49a0be342610eb2834e7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:26:40.605872 10112 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72288a3441fc49a0be342610eb2834e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72288a3441fc49a0be342610eb2834e7" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 41273 } }
I20260808 08:26:40.606276 10112 leader_election.cc:302] T 00000000000000000000000000000000 P 72288a3441fc49a0be342610eb2834e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 72288a3441fc49a0be342610eb2834e7; no voters:
I20260808 08:26:40.606585 10112 leader_election.cc:288] T 00000000000000000000000000000000 P 72288a3441fc49a0be342610eb2834e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:26:40.606670 10119 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72288a3441fc49a0be342610eb2834e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:26:40.606622 10117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:26:40.591700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45974 (local address 127.22.235.254:41273)
0808 08:26:40.592124 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0808 08:26:40.592141 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 08:26:40.592177 (+ 36us) server_negotiation.cc:407] Connection header received
0808 08:26:40.592978 (+ 801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:26:40.593008 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:26:40.593349 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:26:40.593677 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:26:40.594524 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:26:40.595507 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:26:40.596373 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:26:40.597124 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:26:40.597678 (+ 554us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:26:40.600137 (+ 2459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:26:40.600144 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:26:40.600160 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:26:40.600191 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:26:40.602851 (+ 2660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:26:40.603353 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:26:40.603359 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:26:40.603364 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:26:40.603505 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:26:40.603977 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:26:40.603981 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:26:40.603982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:26:40.604550 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:26:40.604960 (+ 410us) server_negotiation.cc:1035] Waiting for connection context
0808 08:26:40.605144 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 08:26:40.605462 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":123,"threads_started":1}
I20260808 08:26:40.606918 10119 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72288a3441fc49a0be342610eb2834e7 [term 1 LEADER]: Becoming Leader. State: Replica: 72288a3441fc49a0be342610eb2834e7, State: Running, Role: LEADER
I20260808 08:26:40.608248 10112 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72288a3441fc49a0be342610eb2834e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:26:40.608270 10119 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72288a3441fc49a0be342610eb2834e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72288a3441fc49a0be342610eb2834e7" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 41273 } }
I20260808 08:26:40.610350 10122 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72288a3441fc49a0be342610eb2834e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "72288a3441fc49a0be342610eb2834e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72288a3441fc49a0be342610eb2834e7" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 41273 } } }
I20260808 08:26:40.610472 10122 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72288a3441fc49a0be342610eb2834e7 [sys.catalog]: This master's current role is: LEADER
I20260808 08:26:40.610911 10112 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 08:26:40.611218 10123 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72288a3441fc49a0be342610eb2834e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72288a3441fc49a0be342610eb2834e7. Latest consensus state: current_term: 1 leader_uuid: "72288a3441fc49a0be342610eb2834e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72288a3441fc49a0be342610eb2834e7" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 41273 } } }
I20260808 08:26:40.611313 10123 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72288a3441fc49a0be342610eb2834e7 [sys.catalog]: This master's current role is: LEADER
I20260808 08:26:40.612263 10143 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:26:40.618963 10143 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:26:40.626226 10143 catalog_manager.cc:1347] Generated new cluster ID: 64247f4d1b2846c5b0ccd724a4575841
I20260808 08:26:40.626292 10143 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:26:40.644127 10143 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:26:40.645282 10143 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:26:40.648980 10143 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 72288a3441fc49a0be342610eb2834e7: Generated new TSK 0
I20260808 08:26:40.649631 10143 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.235.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab.
Entry for principal kudu/127.22.235.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab.
I20260808 08:26:41.432272 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.235.212:38435/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.193:0
--local_ip_for_outbound_sockets=127.22.235.193
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:41273
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab
--principal=kudu/127.22.235.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:45883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:26:41.587159 10185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:26:41.587411 10185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:26:41.587472 10185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:26:41.591990 10185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:26:41.592139 10185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:26:41.592281 10185 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.193
I20260808 08:26:41.599964 10185 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:45883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.22.235.212:38435/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab
--principal=kudu/127.22.235.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:41273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10185
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.193
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:26:41.601248 10185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:26:41.602409 10185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:26:41.613180 10198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:26:41.613397 10199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:26:41.613693 10185 server_base.cc:1034] running on GCE node
W20260808 08:26:41.613708 10201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:26:41.614266 10185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:26:41.615021 10185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:26:41.616259 10185 hybrid_clock.cc:648] HybridClock initialized: now 1786177601616231 us; error 42 us; skew 500 ppm
Aug 08 08:26:41 dist-test-slave-f7th krb5kdc[9233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177601, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:26:41.620082 10185 init.cc:377] Logged in from keytab as kudu/127.22.235.193@KRBTEST.COM (short username kudu)
I20260808 08:26:41.621206 10185 webserver.cc:460] Webserver started at http://127.22.235.193:33715/ using document root <none> and password file <none>
I20260808 08:26:41.621933 10185 fs_manager.cc:359] Metadata directory not provided
I20260808 08:26:41.622025 10185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:26:41.622355 10185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 08:26:41 dist-test-slave-f7th krb5kdc[9233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177601, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for HTTP/127.22.235.212@KRBTEST.COM
I20260808 08:26:41.638849 10185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-0/data/instance:
uuid: "a3207fc803334a0a99e050b6b1fed0e8"
format_stamp: "Formatted at 2026-08-08 08:26:41 on dist-test-slave-f7th"
server_key: "a231318d5c69600edfb09ac144b511c3"
server_key_iv: "5a8505330bb92d95ca4a7b876e44eba6"
server_key_version: "kuduclusterkey@0"
I20260808 08:26:41.639596 10185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance:
uuid: "a3207fc803334a0a99e050b6b1fed0e8"
format_stamp: "Formatted at 2026-08-08 08:26:41 on dist-test-slave-f7th"
server_key: "a231318d5c69600edfb09ac144b511c3"
server_key_iv: "5a8505330bb92d95ca4a7b876e44eba6"
server_key_version: "kuduclusterkey@0"
I20260808 08:26:41.646068 10185 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260808 08:26:41.678294 10218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:26:41.681349 10185 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260808 08:26:41.681579 10185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-0/wal
uuid: "a3207fc803334a0a99e050b6b1fed0e8"
format_stamp: "Formatted at 2026-08-08 08:26:41 on dist-test-slave-f7th"
server_key: "a231318d5c69600edfb09ac144b511c3"
server_key_iv: "5a8505330bb92d95ca4a7b876e44eba6"
server_key_version: "kuduclusterkey@0"
I20260808 08:26:41.681818 10185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:26:41.712097 10185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:26:41.715866 10185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:26:41.716128 10185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:26:41.716996 10185 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:26:41.718253 10185 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:26:41.718379 10185 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:26:41.718511 10185 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:26:41.718601 10185 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:26:41.732316 10185 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.193:41869
I20260808 08:26:41.733744 10185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
I20260808 08:26:41.733769 10417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.193:41869 every 8 connection(s)
I20260808 08:26:41.741706 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 10185
I20260808 08:26:41.741832 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance
I20260808 08:26:41.742159 23471 external_mini_cluster.cc:1428] Setting key 881b1ba776434a24f59ab0eb6e9f3be9
Aug 08 08:26:41 dist-test-slave-f7th krb5kdc[9233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177601, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.235.194@KRBTEST.COM; defaulting to no policy
I20260808 08:26:41.755872 10423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:26:41.736833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:49635 (local address 127.22.235.254:41273)
0808 08:26:41.740826 (+ 3993us) server_negotiation.cc:206] Beginning negotiation
0808 08:26:41.740830 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:26:41.741211 (+ 381us) server_negotiation.cc:407] Connection header received
0808 08:26:41.742709 (+ 1498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:26:41.742713 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:26:41.742773 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:26:41.742869 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:26:41.744383 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:26:41.745138 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:26:41.746749 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:26:41.747091 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:26:41.747182 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:26:41.750524 (+ 3342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:26:41.750528 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:26:41.750531 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:26:41.750558 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:26:41.752480 (+ 1922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:26:41.753143 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:26:41.753146 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:26:41.753148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:26:41.753243 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:26:41.753659 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:26:41.753662 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:26:41.753664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:26:41.753906 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:26:41.754005 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 08:26:41.755593 (+ 1588us) server_negotiation.cc:299] Negotiation successful
0808 08:26:41.755720 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3896,"thread_start_us":92,"threads_started":1}
I20260808 08:26:41.756062 10424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:26:41.737264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:41273 (local address 127.22.235.193:49635)
0808 08:26:41.741056 (+ 3792us) negotiation.cc:107] Waiting for socket to connect
0808 08:26:41.741097 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 08:26:41.742355 (+ 1258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:26:41.743172 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:26:41.743181 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:26:41.743529 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:26:41.744207 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:26:41.744229 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:26:41.745298 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:26:41.745301 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:26:41.746612 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:26:41.746624 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:26:41.747356 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:26:41.747360 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:26:41.747622 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:26:41.748424 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:26:41.748444 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:26:41.750327 (+ 1883us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:26:41.752645 (+ 2318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:26:41.752651 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:26:41.752667 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:26:41.753022 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:26:41.753350 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:26:41.753353 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:26:41.753354 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:26:41.753539 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:26:41.754063 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:26:41.754070 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:26:41.754404 (+ 334us) client_negotiation.cc:769] Sending connection context
0808 08:26:41.754618 (+ 214us) client_negotiation.cc:240] Negotiation successful
0808 08:26:41.754962 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3609,"thread_start_us":113,"threads_started":1}
I20260808 08:26:41.757544 10420 heartbeater.cc:347] Connected to a master server at 127.22.235.254:41273
I20260808 08:26:41.757886 10420 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.194@KRBTEST.COM" created.
I20260808 08:26:41.758527 10420 heartbeater.cc:511] Master 127.22.235.254:41273 requested a full tablet report, sending...
I20260808 08:26:41.760326 10047 ts_manager.cc:194] Registered new tserver with Master: a3207fc803334a0a99e050b6b1fed0e8 (127.22.235.193:41869)
I20260808 08:26:41.761912 10047 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.193@KRBTEST.COM'} at 127.22.235.193:49635
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab.
Entry for principal kudu/127.22.235.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab.
I20260808 08:26:41.778852 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.235.212:38435/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.194:0
--local_ip_for_outbound_sockets=127.22.235.194
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:41273
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab
--principal=kudu/127.22.235.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:45883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 08:26:41 dist-test-slave-f7th krb5kdc[9233](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786177601, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 08:26:41.949282 10430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:26:41.949700 10430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:26:41.949790 10430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:26:41.955657 10430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:26:41.955741 10430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:26:41.955827 10430 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.194
I20260808 08:26:41.963734 10430 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:45883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.22.235.212:38435/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab
--principal=kudu/127.22.235.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:41273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10430
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.194
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:26:41.965489 10430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:26:41.966959 10430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:26:41.980903 10443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:26:41.981498 10439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:26:41.982471 10441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:26:41.982757 10430 server_base.cc:1034] running on GCE node
I20260808 08:26:41.983279 10430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:26:41.984128 10430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:26:41.987406 10430 hybrid_clock.cc:648] HybridClock initialized: now 1786177601987357 us; error 55 us; skew 500 ppm
Aug 08 08:26:41 dist-test-slave-f7th krb5kdc[9233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177601, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:26:41.991842 10430 init.cc:377] Logged in from keytab as kudu/127.22.235.194@KRBTEST.COM (short username kudu)
I20260808 08:26:41.993062 10430 webserver.cc:460] Webserver started at http://127.22.235.194:36865/ using document root <none> and password file <none>
I20260808 08:26:41.993831 10430 fs_manager.cc:359] Metadata directory not provided
I20260808 08:26:41.993969 10430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:26:41.994278 10430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 08:26:41 dist-test-slave-f7th krb5kdc[9233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177601, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.194@KRBTEST.COM for HTTP/127.22.235.212@KRBTEST.COM
I20260808 08:26:42.019761 10430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-1/data/instance:
uuid: "0868c851b4c54667bb8fe2d1ea430e30"
format_stamp: "Formatted at 2026-08-08 08:26:42 on dist-test-slave-f7th"
server_key: "fa31ed8d7eca1b63a31ca077eb733a3f"
server_key_iv: "871f08a7c6e7ebf1f70790ffd905031a"
server_key_version: "kuduclusterkey@0"
I20260808 08:26:42.021057 10430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance:
uuid: "0868c851b4c54667bb8fe2d1ea430e30"
format_stamp: "Formatted at 2026-08-08 08:26:42 on dist-test-slave-f7th"
server_key: "fa31ed8d7eca1b63a31ca077eb733a3f"
server_key_iv: "871f08a7c6e7ebf1f70790ffd905031a"
server_key_version: "kuduclusterkey@0"
I20260808 08:26:42.026973 10430 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 08:26:42.047883 10454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:26:42.049650 10430 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260808 08:26:42.049791 10430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-1/wal
uuid: "0868c851b4c54667bb8fe2d1ea430e30"
format_stamp: "Formatted at 2026-08-08 08:26:42 on dist-test-slave-f7th"
server_key: "fa31ed8d7eca1b63a31ca077eb733a3f"
server_key_iv: "871f08a7c6e7ebf1f70790ffd905031a"
server_key_version: "kuduclusterkey@0"
I20260808 08:26:42.049932 10430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:26:42.068135 10430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:26:42.074705 10430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:26:42.075052 10430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:26:42.076175 10430 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:26:42.078464 10430 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:26:42.078586 10430 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:26:42.078681 10430 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:26:42.078737 10430 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:26:42.096359 10430 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.194:37281
I20260808 08:26:42.097718 10430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
I20260808 08:26:42.098066 10634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.194:37281 every 8 connection(s)
I20260808 08:26:42.106351 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 10430
I20260808 08:26:42.106498 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance
I20260808 08:26:42.106824 23471 external_mini_cluster.cc:1428] Setting key d01bc7a754e0314989368a5dc1591015
Aug 08 08:26:42 dist-test-slave-f7th krb5kdc[9233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177601, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.194@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:26:42.116734 10423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:26:42.100767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:49285 (local address 127.22.235.254:41273)
0808 08:26:42.100908 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 08:26:42.100913 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:26:42.103082 (+ 2169us) server_negotiation.cc:407] Connection header received
0808 08:26:42.104143 (+ 1061us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:26:42.104147 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:26:42.104208 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:26:42.104283 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:26:42.105800 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:26:42.106641 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:26:42.108673 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:26:42.109041 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:26:42.109121 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:26:42.112861 (+ 3740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:26:42.112865 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:26:42.112869 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:26:42.112898 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:26:42.114632 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:26:42.115189 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:26:42.115192 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:26:42.115193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:26:42.115270 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:26:42.115598 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:26:42.115601 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:26:42.115603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:26:42.115783 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:26:42.115871 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 08:26:42.116402 (+ 531us) server_negotiation.cc:299] Negotiation successful
0808 08:26:42.116540 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 08:26:42.117595 10638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:26:42.101264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:41273 (local address 127.22.235.194:49285)
0808 08:26:42.102929 (+ 1665us) negotiation.cc:107] Waiting for socket to connect
0808 08:26:42.102976 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 08:26:42.103910 (+ 934us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:26:42.104477 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:26:42.104484 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:26:42.104857 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:26:42.105607 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:26:42.105631 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:26:42.106781 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:26:42.106784 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:26:42.108507 (+ 1723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:26:42.108519 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:26:42.109320 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:26:42.109324 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:26:42.109556 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:26:42.110953 (+ 1397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:26:42.110975 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:26:42.112653 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:26:42.114752 (+ 2099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:26:42.114758 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:26:42.114776 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:26:42.115079 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:26:42.115357 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:26:42.115360 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:26:42.115363 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:26:42.115500 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:26:42.115883 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:26:42.115898 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:26:42.116148 (+ 250us) client_negotiation.cc:769] Sending connection context
0808 08:26:42.116317 (+ 169us) client_negotiation.cc:240] Negotiation successful
0808 08:26:42.116670 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"thread_start_us":189,"threads_started":1}
I20260808 08:26:42.118736 10635 heartbeater.cc:347] Connected to a master server at 127.22.235.254:41273
I20260808 08:26:42.118993 10635 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.22.235.195@KRBTEST.COM; defaulting to no policy
I20260808 08:26:42.119524 10635 heartbeater.cc:511] Master 127.22.235.254:41273 requested a full tablet report, sending...
I20260808 08:26:42.120584 10047 ts_manager.cc:194] Registered new tserver with Master: 0868c851b4c54667bb8fe2d1ea430e30 (127.22.235.194:37281)
I20260808 08:26:42.121326 10047 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.194@KRBTEST.COM'} at 127.22.235.194:49285
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab.
Entry for principal kudu/127.22.235.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab.
I20260808 08:26:42.140057 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.235.212:38435/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.195:0
--local_ip_for_outbound_sockets=127.22.235.195
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:41273
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab
--principal=kudu/127.22.235.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:45883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 08:26:42 dist-test-slave-f7th krb5kdc[9233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177601, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for HTTP/127.22.235.212@KRBTEST.COM
W20260808 08:26:42.326608 10642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:26:42.327301 10642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:26:42.328095 10642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:26:42.333348 10642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:26:42.333482 10642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:26:42.333608 10642 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.195
I20260808 08:26:42.341625 10642 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:45883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.22.235.212:38435/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab
--principal=kudu/127.22.235.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:41273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10642
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.195
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:26:42.343529 10642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:26:42.345047 10642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:26:42.360460 10662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:26:42.360514 10655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:26:42.361155 10642 server_base.cc:1034] running on GCE node
W20260808 08:26:42.361014 10658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:26:42.361902 10642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:26:42.362720 10642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:26:42.365978 10642 hybrid_clock.cc:648] HybridClock initialized: now 1786177602365904 us; error 90 us; skew 500 ppm
Aug 08 08:26:42 dist-test-slave-f7th krb5kdc[9233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177602, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:26:42.371758 10642 init.cc:377] Logged in from keytab as kudu/127.22.235.195@KRBTEST.COM (short username kudu)
I20260808 08:26:42.372972 10642 webserver.cc:460] Webserver started at http://127.22.235.195:39091/ using document root <none> and password file <none>
I20260808 08:26:42.373760 10642 fs_manager.cc:359] Metadata directory not provided
I20260808 08:26:42.373920 10642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:26:42.374209 10642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 08:26:42 dist-test-slave-f7th krb5kdc[9233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177602, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.195@KRBTEST.COM for HTTP/127.22.235.212@KRBTEST.COM
I20260808 08:26:42.391139 10642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-2/data/instance:
uuid: "b46820c4786b4233ac39e87cce230b9d"
format_stamp: "Formatted at 2026-08-08 08:26:42 on dist-test-slave-f7th"
server_key: "797972b1cb3f60109d1ad72d6c0881a6"
server_key_iv: "a45fe4e6b32a8dd1832fc7c082e58f9a"
server_key_version: "kuduclusterkey@0"
I20260808 08:26:42.392329 10642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance:
uuid: "b46820c4786b4233ac39e87cce230b9d"
format_stamp: "Formatted at 2026-08-08 08:26:42 on dist-test-slave-f7th"
server_key: "797972b1cb3f60109d1ad72d6c0881a6"
server_key_iv: "a45fe4e6b32a8dd1832fc7c082e58f9a"
server_key_version: "kuduclusterkey@0"
I20260808 08:26:42.403712 10642 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.000s sys 0.003s
I20260808 08:26:42.430544 10676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:26:42.432245 10642 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 08:26:42.432385 10642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-2/wal
uuid: "b46820c4786b4233ac39e87cce230b9d"
format_stamp: "Formatted at 2026-08-08 08:26:42 on dist-test-slave-f7th"
server_key: "797972b1cb3f60109d1ad72d6c0881a6"
server_key_iv: "a45fe4e6b32a8dd1832fc7c082e58f9a"
server_key_version: "kuduclusterkey@0"
I20260808 08:26:42.432516 10642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:26:42.478318 10642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:26:42.482093 10642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:26:42.482309 10642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:26:42.483239 10642 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:26:42.484515 10642 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:26:42.484584 10642 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:26:42.484664 10642 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:26:42.484696 10642 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:26:42.499622 10642 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.195:44405
I20260808 08:26:42.499845 10880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.195:44405 every 8 connection(s)
I20260808 08:26:42.500953 10642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
I20260808 08:26:42.503537 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 10642
I20260808 08:26:42.503660 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance
I20260808 08:26:42.504025 23471 external_mini_cluster.cc:1428] Setting key 5353589be1154a3ab730fd074622ab8c
Aug 08 08:26:42 dist-test-slave-f7th krb5kdc[9233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177602, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.195@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:26:42.521122 10423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:26:42.505304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:43551 (local address 127.22.235.254:41273)
0808 08:26:42.505434 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 08:26:42.505438 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:26:42.506918 (+ 1480us) server_negotiation.cc:407] Connection header received
0808 08:26:42.508510 (+ 1592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:26:42.508514 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:26:42.508569 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:26:42.508687 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:26:42.510479 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:26:42.511309 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:26:42.513186 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:26:42.513539 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:26:42.513656 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:26:42.516839 (+ 3183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:26:42.516843 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:26:42.516846 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:26:42.516876 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:26:42.518662 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:26:42.519306 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:26:42.519311 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:26:42.519313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:26:42.519413 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:26:42.519823 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:26:42.519827 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:26:42.519829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:26:42.520073 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:26:42.520183 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 08:26:42.520824 (+ 641us) server_negotiation.cc:299] Negotiation successful
0808 08:26:42.520953 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 08:26:42.522213 10886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:26:42.504104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:41273 (local address 127.22.235.195:43551)
0808 08:26:42.506714 (+ 2610us) negotiation.cc:107] Waiting for socket to connect
0808 08:26:42.506775 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 08:26:42.508267 (+ 1492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:26:42.509032 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:26:42.509050 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:26:42.509437 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:26:42.510292 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:26:42.510318 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:26:42.511453 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:26:42.511457 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:26:42.513041 (+ 1584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:26:42.513053 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:26:42.513710 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:26:42.513714 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:26:42.513933 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:26:42.514670 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:26:42.514691 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:26:42.516609 (+ 1918us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:26:42.518792 (+ 2183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:26:42.518807 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:26:42.518825 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:26:42.519175 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:26:42.519511 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:26:42.519514 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:26:42.519516 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:26:42.519693 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:26:42.520179 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:26:42.520185 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:26:42.520507 (+ 322us) client_negotiation.cc:769] Sending connection context
0808 08:26:42.520734 (+ 227us) client_negotiation.cc:240] Negotiation successful
0808 08:26:42.521083 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2249,"thread_start_us":196,"threads_started":1}
I20260808 08:26:42.523504 10883 heartbeater.cc:347] Connected to a master server at 127.22.235.254:41273
I20260808 08:26:42.523784 10883 heartbeater.cc:464] Registering TS with master...
I20260808 08:26:42.524420 10883 heartbeater.cc:511] Master 127.22.235.254:41273 requested a full tablet report, sending...
I20260808 08:26:42.525689 10047 ts_manager.cc:194] Registered new tserver with Master: b46820c4786b4233ac39e87cce230b9d (127.22.235.195:44405)
I20260808 08:26:42.526455 10047 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.195@KRBTEST.COM'} at 127.22.235.195:43551
I20260808 08:26:42.530417 23471 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 08:26:42.542662 10423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:26:42.533361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50028 (local address 127.22.235.254:41273)
0808 08:26:42.533490 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 08:26:42.533494 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:26:42.533536 (+ 42us) server_negotiation.cc:407] Connection header received
0808 08:26:42.533684 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:26:42.533688 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:26:42.533740 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:26:42.533841 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:26:42.534706 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:26:42.535512 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:26:42.536703 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:26:42.537054 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:26:42.537172 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:26:42.538933 (+ 1761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:26:42.538938 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:26:42.538940 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:26:42.538969 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:26:42.540753 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:26:42.541232 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:26:42.541235 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:26:42.541237 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:26:42.541334 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:26:42.541605 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:26:42.541609 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:26:42.541611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:26:42.541853 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:26:42.541986 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 08:26:42.542217 (+ 231us) server_negotiation.cc:299] Negotiation successful
0808 08:26:42.542343 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 08:26:42.546277 10047 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50028:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 08:26:42.550760 10047 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 08:26:42.574329 10909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:26:42.562267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.194:37281 (local address 127.0.0.1:50544)
0808 08:26:42.564802 (+ 2535us) negotiation.cc:107] Waiting for socket to connect
0808 08:26:42.564815 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:26:42.564946 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:26:42.565876 (+ 930us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:26:42.565879 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:26:42.565922 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:26:42.567007 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:26:42.567019 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:26:42.570141 (+ 3122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:26:42.570144 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:26:42.571545 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:26:42.571557 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:26:42.572979 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:26:42.572983 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:26:42.573136 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:26:42.573929 (+ 793us) client_negotiation.cc:769] Sending connection context
0808 08:26:42.574040 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 08:26:42.574167 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2466,"mutex_wait_us":852,"spinlock_wait_cycles":5248,"thread_start_us":151,"threads_started":1}
I20260808 08:26:42.574880 10920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:26:42.561998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.195:44405 (local address 127.0.0.1:60108)
0808 08:26:42.564029 (+ 2031us) negotiation.cc:107] Waiting for socket to connect
0808 08:26:42.564084 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 08:26:42.564233 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:26:42.567216 (+ 2983us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:26:42.567218 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:26:42.567237 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:26:42.567359 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:26:42.567366 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:26:42.570255 (+ 2889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:26:42.570258 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:26:42.573304 (+ 3046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:26:42.573314 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:26:42.574524 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:26:42.574528 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:26:42.574609 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:26:42.574630 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 08:26:42.574696 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 08:26:42.574756 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1764,"mutex_wait_us":1690,"thread_start_us":127,"threads_started":1}
I20260808 08:26:42.575460 10916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:26:42.562539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50544 (local address 127.22.235.194:37281)
0808 08:26:42.563669 (+ 1130us) server_negotiation.cc:206] Beginning negotiation
0808 08:26:42.563685 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 08:26:42.565296 (+ 1611us) server_negotiation.cc:407] Connection header received
0808 08:26:42.565380 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:26:42.565385 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:26:42.565560 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:26:42.565706 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:26:42.567491 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:26:42.568731 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:26:42.571746 (+ 3015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:26:42.572642 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:26:42.572759 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:26:42.572919 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 08:26:42.575169 (+ 2250us) server_negotiation.cc:299] Negotiation successful
0808 08:26:42.575280 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1007,"thread_start_us":119,"threads_started":1}
I20260808 08:26:42.575847 10915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:26:42.562973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:41869 (local address 127.0.0.1:48926)
0808 08:26:42.565111 (+ 2138us) negotiation.cc:107] Waiting for socket to connect
0808 08:26:42.565122 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 08:26:42.565189 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:26:42.565871 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:26:42.565877 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:26:42.565920 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:26:42.567128 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:26:42.567134 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:26:42.573079 (+ 5945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:26:42.573083 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:26:42.574287 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:26:42.574308 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:26:42.575413 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:26:42.575417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:26:42.575583 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:26:42.575608 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 08:26:42.575656 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 08:26:42.575718 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2083,"mutex_wait_us":1054,"spinlock_wait_cycles":11904,"thread_start_us":164,"threads_started":1}
I20260808 08:26:42.576750 10925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:26:42.563156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48926 (local address 127.22.235.193:41869)
0808 08:26:42.565421 (+ 2265us) server_negotiation.cc:206] Beginning negotiation
0808 08:26:42.565428 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 08:26:42.565450 (+ 22us) server_negotiation.cc:407] Connection header received
0808 08:26:42.565522 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:26:42.565527 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:26:42.565702 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:26:42.566125 (+ 423us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:26:42.568890 (+ 2765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:26:42.572941 (+ 4051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:26:42.574460 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:26:42.575271 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:26:42.576269 (+ 998us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:26:42.576412 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 08:26:42.576465 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 08:26:42.576557 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2161,"thread_start_us":101,"threads_started":1}
I20260808 08:26:42.578997 10531 tablet_service.cc:1467] Processing CreateTablet for tablet d39fab9c5d104e929aa10a73e6245697 (DEFAULT_TABLE table=test-table [id=38b2194a141948b1b7fb1e3ee565c6fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:26:42.579859 10314 tablet_service.cc:1467] Processing CreateTablet for tablet d39fab9c5d104e929aa10a73e6245697 (DEFAULT_TABLE table=test-table [id=38b2194a141948b1b7fb1e3ee565c6fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:26:42.581175 10314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d39fab9c5d104e929aa10a73e6245697. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:26:42.584367 10912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:26:42.562050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60108 (local address 127.22.235.195:44405)
0808 08:26:42.564547 (+ 2497us) server_negotiation.cc:206] Beginning negotiation
0808 08:26:42.564596 (+ 49us) server_negotiation.cc:399] Waiting for connection header
0808 08:26:42.564659 (+ 63us) server_negotiation.cc:407] Connection header received
0808 08:26:42.564754 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:26:42.564760 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:26:42.564989 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:26:42.565143 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:26:42.569114 (+ 3971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:26:42.570136 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:26:42.573468 (+ 3332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:26:42.574214 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:26:42.574333 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:26:42.574462 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 08:26:42.584053 (+ 9591us) server_negotiation.cc:299] Negotiation successful
0808 08:26:42.584172 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2393,"thread_start_us":127,"threads_started":1}
I20260808 08:26:42.585245 10531 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d39fab9c5d104e929aa10a73e6245697. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:26:42.587337 10757 tablet_service.cc:1467] Processing CreateTablet for tablet d39fab9c5d104e929aa10a73e6245697 (DEFAULT_TABLE table=test-table [id=38b2194a141948b1b7fb1e3ee565c6fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:26:42.588485 10757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d39fab9c5d104e929aa10a73e6245697. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:26:42.593849 10927 tablet_bootstrap.cc:492] T d39fab9c5d104e929aa10a73e6245697 P a3207fc803334a0a99e050b6b1fed0e8: Bootstrap starting.
I20260808 08:26:42.596755 10927 tablet_bootstrap.cc:654] T d39fab9c5d104e929aa10a73e6245697 P a3207fc803334a0a99e050b6b1fed0e8: Neither blocks nor log segments found. Creating new log.
I20260808 08:26:42.597841 10927 log.cc:824] T d39fab9c5d104e929aa10a73e6245697 P a3207fc803334a0a99e050b6b1fed0e8: Log is configured to *not* fsync() on all Append() calls
I20260808 08:26:42.599989 10927 tablet_bootstrap.cc:492] T d39fab9c5d104e929aa10a73e6245697 P a3207fc803334a0a99e050b6b1fed0e8: No bootstrap required, opened a new log
I20260808 08:26:42.600176 10927 ts_tablet_manager.cc:1397] T d39fab9c5d104e929aa10a73e6245697 P a3207fc803334a0a99e050b6b1fed0e8: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.006s
I20260808 08:26:42.605415 10927 raft_consensus.cc:348] T d39fab9c5d104e929aa10a73e6245697 P a3207fc803334a0a99e050b6b1fed0e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b46820c4786b4233ac39e87cce230b9d" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44405 } } peers { permanent_uuid: "a3207fc803334a0a99e050b6b1fed0e8" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41869 } } peers { permanent_uuid: "0868c851b4c54667bb8fe2d1ea430e30" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 37281 } }
I20260808 08:26:42.605681 10927 raft_consensus.cc:374] T d39fab9c5d104e929aa10a73e6245697 P a3207fc803334a0a99e050b6b1fed0e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:26:42.605736 10927 raft_consensus.cc:729] T d39fab9c5d104e929aa10a73e6245697 P a3207fc803334a0a99e050b6b1fed0e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3207fc803334a0a99e050b6b1fed0e8, State: Initialized, Role: FOLLOWER
I20260808 08:26:42.606408 10927 consensus_queue.cc:260] T d39fab9c5d104e929aa10a73e6245697 P a3207fc803334a0a99e050b6b1fed0e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b46820c4786b4233ac39e87cce230b9d" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44405 } } peers { permanent_uuid: "a3207fc803334a0a99e050b6b1fed0e8" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41869 } } peers { permanent_uuid: "0868c851b4c54667bb8fe2d1ea430e30" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 37281 } }
I20260808 08:26:42.607666 10927 ts_tablet_manager.cc:1428] T d39fab9c5d104e929aa10a73e6245697 P a3207fc803334a0a99e050b6b1fed0e8: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260808 08:26:42.608054 10420 heartbeater.cc:503] Master 127.22.235.254:41273 was elected leader, sending a full tablet report...
I20260808 08:26:42.610600 10928 tablet_bootstrap.cc:492] T d39fab9c5d104e929aa10a73e6245697 P 0868c851b4c54667bb8fe2d1ea430e30: Bootstrap starting.
I20260808 08:26:42.612989 10932 tablet_bootstrap.cc:492] T d39fab9c5d104e929aa10a73e6245697 P b46820c4786b4233ac39e87cce230b9d: Bootstrap starting.
I20260808 08:26:42.614562 10928 tablet_bootstrap.cc:654] T d39fab9c5d104e929aa10a73e6245697 P 0868c851b4c54667bb8fe2d1ea430e30: Neither blocks nor log segments found. Creating new log.
I20260808 08:26:42.615592 10928 log.cc:824] T d39fab9c5d104e929aa10a73e6245697 P 0868c851b4c54667bb8fe2d1ea430e30: Log is configured to *not* fsync() on all Append() calls
I20260808 08:26:42.616871 10932 tablet_bootstrap.cc:654] T d39fab9c5d104e929aa10a73e6245697 P b46820c4786b4233ac39e87cce230b9d: Neither blocks nor log segments found. Creating new log.
I20260808 08:26:42.617883 10932 log.cc:824] T d39fab9c5d104e929aa10a73e6245697 P b46820c4786b4233ac39e87cce230b9d: Log is configured to *not* fsync() on all Append() calls
I20260808 08:26:42.617933 10928 tablet_bootstrap.cc:492] T d39fab9c5d104e929aa10a73e6245697 P 0868c851b4c54667bb8fe2d1ea430e30: No bootstrap required, opened a new log
I20260808 08:26:42.618125 10928 ts_tablet_manager.cc:1397] T d39fab9c5d104e929aa10a73e6245697 P 0868c851b4c54667bb8fe2d1ea430e30: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260808 08:26:42.622298 10928 raft_consensus.cc:348] T d39fab9c5d104e929aa10a73e6245697 P 0868c851b4c54667bb8fe2d1ea430e30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b46820c4786b4233ac39e87cce230b9d" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44405 } } peers { permanent_uuid: "a3207fc803334a0a99e050b6b1fed0e8" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41869 } } peers { permanent_uuid: "0868c851b4c54667bb8fe2d1ea430e30" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 37281 } }
I20260808 08:26:42.622566 10928 raft_consensus.cc:374] T d39fab9c5d104e929aa10a73e6245697 P 0868c851b4c54667bb8fe2d1ea430e30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:26:42.622654 10928 raft_consensus.cc:729] T d39fab9c5d104e929aa10a73e6245697 P 0868c851b4c54667bb8fe2d1ea430e30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0868c851b4c54667bb8fe2d1ea430e30, State: Initialized, Role: FOLLOWER
I20260808 08:26:42.623114 10928 consensus_queue.cc:260] T d39fab9c5d104e929aa10a73e6245697 P 0868c851b4c54667bb8fe2d1ea430e30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b46820c4786b4233ac39e87cce230b9d" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44405 } } peers { permanent_uuid: "a3207fc803334a0a99e050b6b1fed0e8" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41869 } } peers { permanent_uuid: "0868c851b4c54667bb8fe2d1ea430e30" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 37281 } }
I20260808 08:26:42.623772 10635 heartbeater.cc:503] Master 127.22.235.254:41273 was elected leader, sending a full tablet report...
I20260808 08:26:42.625162 10928 ts_tablet_manager.cc:1428] T d39fab9c5d104e929aa10a73e6245697 P 0868c851b4c54667bb8fe2d1ea430e30: Time spent starting tablet: real 0.007s user 0.003s sys 0.002s
I20260808 08:26:42.627506 10932 tablet_bootstrap.cc:492] T d39fab9c5d104e929aa10a73e6245697 P b46820c4786b4233ac39e87cce230b9d: No bootstrap required, opened a new log
I20260808 08:26:42.627727 10932 ts_tablet_manager.cc:1397] T d39fab9c5d104e929aa10a73e6245697 P b46820c4786b4233ac39e87cce230b9d: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.000s
I20260808 08:26:42.632282 10932 raft_consensus.cc:348] T d39fab9c5d104e929aa10a73e6245697 P b46820c4786b4233ac39e87cce230b9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b46820c4786b4233ac39e87cce230b9d" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44405 } } peers { permanent_uuid: "a3207fc803334a0a99e050b6b1fed0e8" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41869 } } peers { permanent_uuid: "0868c851b4c54667bb8fe2d1ea430e30" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 37281 } }
I20260808 08:26:42.632573 10932 raft_consensus.cc:374] T d39fab9c5d104e929aa10a73e6245697 P b46820c4786b4233ac39e87cce230b9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:26:42.632659 10932 raft_consensus.cc:729] T d39fab9c5d104e929aa10a73e6245697 P b46820c4786b4233ac39e87cce230b9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b46820c4786b4233ac39e87cce230b9d, State: Initialized, Role: FOLLOWER
I20260808 08:26:42.633103 10932 consensus_queue.cc:260] T d39fab9c5d104e929aa10a73e6245697 P b46820c4786b4233ac39e87cce230b9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b46820c4786b4233ac39e87cce230b9d" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44405 } } peers { permanent_uuid: "a3207fc803334a0a99e050b6b1fed0e8" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41869 } } peers { permanent_uuid: "0868c851b4c54667bb8fe2d1ea430e30" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 37281 } }
I20260808 08:26:42.633759 10883 heartbeater.cc:503] Master 127.22.235.254:41273 was elected leader, sending a full tablet report...
I20260808 08:26:42.634152 10932 ts_tablet_manager.cc:1428] T d39fab9c5d104e929aa10a73e6245697 P b46820c4786b4233ac39e87cce230b9d: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
Aug 08 08:26:42 dist-test-slave-f7th krb5kdc[9233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177601, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for HTTP/127.22.235.212@KRBTEST.COM
I20260808 08:26:42.682601 10936 raft_consensus.cc:482] T d39fab9c5d104e929aa10a73e6245697 P 0868c851b4c54667bb8fe2d1ea430e30 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 08:26:42.682854 10936 raft_consensus.cc:504] T d39fab9c5d104e929aa10a73e6245697 P 0868c851b4c54667bb8fe2d1ea430e30 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b46820c4786b4233ac39e87cce230b9d" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44405 } } peers { permanent_uuid: "a3207fc803334a0a99e050b6b1fed0e8" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41869 } } peers { permanent_uuid: "0868c851b4c54667bb8fe2d1ea430e30" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 37281 } }
I20260808 08:26:42.684283 10936 leader_election.cc:288] T d39fab9c5d104e929aa10a73e6245697 P 0868c851b4c54667bb8fe2d1ea430e30 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b46820c4786b4233ac39e87cce230b9d (127.22.235.195:44405), a3207fc803334a0a99e050b6b1fed0e8 (127.22.235.193:41869)
I20260808 08:26:42.691046 10912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:26:42.685212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:35131 (local address 127.22.235.195:44405)
0808 08:26:42.685338 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 08:26:42.685343 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:26:42.685644 (+ 301us) server_negotiation.cc:407] Connection header received
0808 08:26:42.685844 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:26:42.685848 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:26:42.685902 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:26:42.685955 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:26:42.686853 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:26:42.687646 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:26:42.689693 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:26:42.690332 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:26:42.690426 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:26:42.690493 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 08:26:42.690834 (+ 341us) server_negotiation.cc:299] Negotiation successful
0808 08:26:42.690913 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 08:26:42.691699 10942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:26:42.684483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.195:44405 (local address 127.22.235.194:35131)
0808 08:26:42.685547 (+ 1064us) negotiation.cc:107] Waiting for socket to connect
0808 08:26:42.685562 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 08:26:42.685692 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:26:42.686122 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:26:42.686124 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:26:42.686535 (+ 411us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:26:42.686680 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:26:42.686689 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:26:42.688015 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:26:42.688018 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:26:42.689544 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:26:42.689557 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:26:42.690592 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:26:42.690595 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:26:42.690696 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:26:42.690719 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 08:26:42.691440 (+ 721us) client_negotiation.cc:240] Negotiation successful
0808 08:26:42.691535 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":990,"thread_start_us":313,"threads_started":1}
I20260808 08:26:42.692497 10795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d39fab9c5d104e929aa10a73e6245697" candidate_uuid: "0868c851b4c54667bb8fe2d1ea430e30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b46820c4786b4233ac39e87cce230b9d" is_pre_election: true
I20260808 08:26:42.692875 10795 raft_consensus.cc:2454] T d39fab9c5d104e929aa10a73e6245697 P b46820c4786b4233ac39e87cce230b9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0868c851b4c54667bb8fe2d1ea430e30 in term 0.
I20260808 08:26:42.693523 10462 leader_election.cc:302] T d39fab9c5d104e929aa10a73e6245697 P 0868c851b4c54667bb8fe2d1ea430e30 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0868c851b4c54667bb8fe2d1ea430e30, b46820c4786b4233ac39e87cce230b9d; no voters:
I20260808 08:26:42.693928 10936 raft_consensus.cc:2781] T d39fab9c5d104e929aa10a73e6245697 P 0868c851b4c54667bb8fe2d1ea430e30 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 08:26:42.694016 10936 raft_consensus.cc:482] T d39fab9c5d104e929aa10a73e6245697 P 0868c851b4c54667bb8fe2d1ea430e30 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 08:26:42.694057 10936 raft_consensus.cc:3037] T d39fab9c5d104e929aa10a73e6245697 P 0868c851b4c54667bb8fe2d1ea430e30 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:26:42.695392 10936 raft_consensus.cc:504] T d39fab9c5d104e929aa10a73e6245697 P 0868c851b4c54667bb8fe2d1ea430e30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b46820c4786b4233ac39e87cce230b9d" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44405 } } peers { permanent_uuid: "a3207fc803334a0a99e050b6b1fed0e8" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41869 } } peers { permanent_uuid: "0868c851b4c54667bb8fe2d1ea430e30" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 37281 } }
I20260808 08:26:42.695763 10936 leader_election.cc:288] T d39fab9c5d104e929aa10a73e6245697 P 0868c851b4c54667bb8fe2d1ea430e30 [CANDIDATE]: Term 1 election: Requested vote from peers b46820c4786b4233ac39e87cce230b9d (127.22.235.195:44405), a3207fc803334a0a99e050b6b1fed0e8 (127.22.235.193:41869)
I20260808 08:26:42.696405 10795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d39fab9c5d104e929aa10a73e6245697" candidate_uuid: "0868c851b4c54667bb8fe2d1ea430e30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b46820c4786b4233ac39e87cce230b9d"
I20260808 08:26:42.696576 10795 raft_consensus.cc:3037] T d39fab9c5d104e929aa10a73e6245697 P b46820c4786b4233ac39e87cce230b9d [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:26:42.697494 10925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:26:42.684851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:55343 (local address 127.22.235.193:41869)
0808 08:26:42.685001 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 08:26:42.685006 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:26:42.689218 (+ 4212us) server_negotiation.cc:407] Connection header received
0808 08:26:42.689426 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:26:42.689430 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:26:42.689485 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:26:42.689534 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:26:42.690331 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:26:42.691095 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:26:42.695470 (+ 4375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:26:42.696089 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:26:42.696186 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:26:42.696236 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 08:26:42.697265 (+ 1029us) server_negotiation.cc:299] Negotiation successful
0808 08:26:42.697369 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 08:26:42.698127 10945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:26:42.685178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:41869 (local address 127.22.235.194:55343)
0808 08:26:42.689120 (+ 3942us) negotiation.cc:107] Waiting for socket to connect
0808 08:26:42.689135 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 08:26:42.689303 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:26:42.689905 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:26:42.689908 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:26:42.689930 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:26:42.690165 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:26:42.690174 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:26:42.691470 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:26:42.691474 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:26:42.695320 (+ 3846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:26:42.695339 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:26:42.696658 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:26:42.696662 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:26:42.696810 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:26:42.696837 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 08:26:42.697907 (+ 1070us) client_negotiation.cc:240] Negotiation successful
0808 08:26:42.697986 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3864,"thread_start_us":92,"threads_started":1}
I20260808 08:26:42.698155 10795 raft_consensus.cc:2454] T d39fab9c5d104e929aa10a73e6245697 P b46820c4786b4233ac39e87cce230b9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0868c851b4c54667bb8fe2d1ea430e30 in term 1.
I20260808 08:26:42.699085 10341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d39fab9c5d104e929aa10a73e6245697" candidate_uuid: "0868c851b4c54667bb8fe2d1ea430e30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3207fc803334a0a99e050b6b1fed0e8" is_pre_election: true
I20260808 08:26:42.699307 10462 leader_election.cc:302] T d39fab9c5d104e929aa10a73e6245697 P 0868c851b4c54667bb8fe2d1ea430e30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0868c851b4c54667bb8fe2d1ea430e30, b46820c4786b4233ac39e87cce230b9d; no voters:
I20260808 08:26:42.699404 10341 raft_consensus.cc:2454] T d39fab9c5d104e929aa10a73e6245697 P a3207fc803334a0a99e050b6b1fed0e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0868c851b4c54667bb8fe2d1ea430e30 in term 0.
I20260808 08:26:42.699515 10936 raft_consensus.cc:2781] T d39fab9c5d104e929aa10a73e6245697 P 0868c851b4c54667bb8fe2d1ea430e30 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:26:42.699738 10936 raft_consensus.cc:686] T d39fab9c5d104e929aa10a73e6245697 P 0868c851b4c54667bb8fe2d1ea430e30 [term 1 LEADER]: Becoming Leader. State: Replica: 0868c851b4c54667bb8fe2d1ea430e30, State: Running, Role: LEADER
I20260808 08:26:42.699852 10339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d39fab9c5d104e929aa10a73e6245697" candidate_uuid: "0868c851b4c54667bb8fe2d1ea430e30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3207fc803334a0a99e050b6b1fed0e8"
I20260808 08:26:42.699968 10339 raft_consensus.cc:3037] T d39fab9c5d104e929aa10a73e6245697 P a3207fc803334a0a99e050b6b1fed0e8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:26:42.700116 10936 consensus_queue.cc:237] T d39fab9c5d104e929aa10a73e6245697 P 0868c851b4c54667bb8fe2d1ea430e30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b46820c4786b4233ac39e87cce230b9d" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44405 } } peers { permanent_uuid: "a3207fc803334a0a99e050b6b1fed0e8" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41869 } } peers { permanent_uuid: "0868c851b4c54667bb8fe2d1ea430e30" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 37281 } }
I20260808 08:26:42.701310 10339 raft_consensus.cc:2454] T d39fab9c5d104e929aa10a73e6245697 P a3207fc803334a0a99e050b6b1fed0e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0868c851b4c54667bb8fe2d1ea430e30 in term 1.
I20260808 08:26:42.703677 10047 catalog_manager.cc:5495] T d39fab9c5d104e929aa10a73e6245697 P 0868c851b4c54667bb8fe2d1ea430e30 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0868c851b4c54667bb8fe2d1ea430e30 (127.22.235.194). New cstate: current_term: 1 leader_uuid: "0868c851b4c54667bb8fe2d1ea430e30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b46820c4786b4233ac39e87cce230b9d" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3207fc803334a0a99e050b6b1fed0e8" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0868c851b4c54667bb8fe2d1ea430e30" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 37281 } health_report { overall_health: HEALTHY } } }
I20260808 08:26:42.729162 10916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:26:42.724111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50548 (local address 127.22.235.194:37281)
0808 08:26:42.724258 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 08:26:42.724264 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 08:26:42.724281 (+ 17us) server_negotiation.cc:407] Connection header received
0808 08:26:42.724422 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:26:42.724425 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:26:42.724504 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:26:42.724641 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:26:42.725052 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:26:42.725885 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:26:42.727218 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:26:42.727555 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:26:42.727663 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:26:42.728056 (+ 393us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:26:42.728669 (+ 613us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 08:26:42.728797 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 08:26:42.728957 (+ 160us) server_negotiation.cc:299] Negotiation successful
0808 08:26:42.729022 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 08:26:42.737560 10795 raft_consensus.cc:1261] T d39fab9c5d104e929aa10a73e6245697 P b46820c4786b4233ac39e87cce230b9d [term 1 FOLLOWER]: Refusing update from remote peer 0868c851b4c54667bb8fe2d1ea430e30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:26:42.738476 10947 consensus_queue.cc:1035] T d39fab9c5d104e929aa10a73e6245697 P 0868c851b4c54667bb8fe2d1ea430e30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b46820c4786b4233ac39e87cce230b9d" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 08:26:42.741389 10421 tablet.cc:2369] T d39fab9c5d104e929aa10a73e6245697 P a3207fc803334a0a99e050b6b1fed0e8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 08:26:42.742276 10339 raft_consensus.cc:1261] T d39fab9c5d104e929aa10a73e6245697 P a3207fc803334a0a99e050b6b1fed0e8 [term 1 FOLLOWER]: Refusing update from remote peer 0868c851b4c54667bb8fe2d1ea430e30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:26:42.742938 10947 consensus_queue.cc:1035] T d39fab9c5d104e929aa10a73e6245697 P 0868c851b4c54667bb8fe2d1ea430e30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3207fc803334a0a99e050b6b1fed0e8" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 41869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:26:42.752171 10954 mvcc.cc:205] Tried to move back new op lower bound from 7316183460802011136 to 7316183460663476224. Current Snapshot: MvccSnapshot[applied={T|T < 7316183460802011136}]
I20260808 08:26:42.755244 10958 mvcc.cc:205] Tried to move back new op lower bound from 7316183460802011136 to 7316183460663476224. Current Snapshot: MvccSnapshot[applied={T|T < 7316183460802011136}]
I20260808 08:26:42.764652 10047 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50028:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:26:42.764892 10047 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50028:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:26:42.767472 10047 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 72288a3441fc49a0be342610eb2834e7: Sending DeleteTablet for 3 replicas of tablet d39fab9c5d104e929aa10a73e6245697
I20260808 08:26:42.768208 10314 tablet_service.cc:1514] Processing DeleteTablet for tablet d39fab9c5d104e929aa10a73e6245697 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:26:42 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:48926
I20260808 08:26:42.768226 10757 tablet_service.cc:1514] Processing DeleteTablet for tablet d39fab9c5d104e929aa10a73e6245697 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:26:42 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:60108
I20260808 08:26:42.768767 10967 tablet_replica.cc:330] T d39fab9c5d104e929aa10a73e6245697 P a3207fc803334a0a99e050b6b1fed0e8: stopping tablet replica
I20260808 08:26:42.769124 10967 raft_consensus.cc:2229] T d39fab9c5d104e929aa10a73e6245697 P a3207fc803334a0a99e050b6b1fed0e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 08:26:42.769209 10531 tablet_service.cc:1514] Processing DeleteTablet for tablet d39fab9c5d104e929aa10a73e6245697 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:26:42 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:50544
I20260808 08:26:42.769511 10967 raft_consensus.cc:2258] T d39fab9c5d104e929aa10a73e6245697 P a3207fc803334a0a99e050b6b1fed0e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:26:42.770231 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 10185
I20260808 08:26:42.784940 10969 tablet_replica.cc:330] T d39fab9c5d104e929aa10a73e6245697 P b46820c4786b4233ac39e87cce230b9d: stopping tablet replica
I20260808 08:26:42.784984 10971 tablet_replica.cc:330] T d39fab9c5d104e929aa10a73e6245697 P 0868c851b4c54667bb8fe2d1ea430e30: stopping tablet replica
I20260808 08:26:42.785427 10971 raft_consensus.cc:2229] T d39fab9c5d104e929aa10a73e6245697 P 0868c851b4c54667bb8fe2d1ea430e30 [term 1 LEADER]: Raft consensus shutting down.
I20260808 08:26:42.785637 10969 raft_consensus.cc:2229] T d39fab9c5d104e929aa10a73e6245697 P b46820c4786b4233ac39e87cce230b9d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 08:26:42.786221 10969 raft_consensus.cc:2258] T d39fab9c5d104e929aa10a73e6245697 P b46820c4786b4233ac39e87cce230b9d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:26:42.788345 10971 raft_consensus.cc:2258] T d39fab9c5d104e929aa10a73e6245697 P 0868c851b4c54667bb8fe2d1ea430e30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:26:42.788501 10969 ts_tablet_manager.cc:1905] T d39fab9c5d104e929aa10a73e6245697 P b46820c4786b4233ac39e87cce230b9d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:26:42.790030 10971 ts_tablet_manager.cc:1905] T d39fab9c5d104e929aa10a73e6245697 P 0868c851b4c54667bb8fe2d1ea430e30: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:26:42.791989 10969 ts_tablet_manager.cc:1918] T d39fab9c5d104e929aa10a73e6245697 P b46820c4786b4233ac39e87cce230b9d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 08:26:42.793079 10971 ts_tablet_manager.cc:1918] T d39fab9c5d104e929aa10a73e6245697 P 0868c851b4c54667bb8fe2d1ea430e30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 08:26:42.793184 10971 log.cc:1196] T d39fab9c5d104e929aa10a73e6245697 P 0868c851b4c54667bb8fe2d1ea430e30: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-1/wal/wals/d39fab9c5d104e929aa10a73e6245697
I20260808 08:26:42.793521 10971 ts_tablet_manager.cc:1939] T d39fab9c5d104e929aa10a73e6245697 P 0868c851b4c54667bb8fe2d1ea430e30: Deleting consensus metadata
I20260808 08:26:42.794879 10022 catalog_manager.cc:4841] TS 0868c851b4c54667bb8fe2d1ea430e30 (127.22.235.194:37281): tablet d39fab9c5d104e929aa10a73e6245697 (table test-table [id=38b2194a141948b1b7fb1e3ee565c6fe]) successfully deleted
I20260808 08:26:42.795127 10969 log.cc:1196] T d39fab9c5d104e929aa10a73e6245697 P b46820c4786b4233ac39e87cce230b9d: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786177493204855-23471-0/minicluster-data/ts-2/wal/wals/d39fab9c5d104e929aa10a73e6245697
I20260808 08:26:42.795567 10969 ts_tablet_manager.cc:1939] T d39fab9c5d104e929aa10a73e6245697 P b46820c4786b4233ac39e87cce230b9d: Deleting consensus metadata
I20260808 08:26:42.796991 10023 catalog_manager.cc:4841] TS b46820c4786b4233ac39e87cce230b9d (127.22.235.195:44405): tablet d39fab9c5d104e929aa10a73e6245697 (table test-table [id=38b2194a141948b1b7fb1e3ee565c6fe]) successfully deleted
W20260808 08:26:42.797140 10021 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.235.193:41869 (error 108)
I20260808 08:26:42.800683 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 10430
I20260808 08:26:42.843307 10915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:26:42.842888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:41869 (local address 127.0.0.1:48936)
0808 08:26:42.843113 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0808 08:26:42.843198 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.193:41869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
W20260808 08:26:42.843704 10021 catalog_manager.cc:4584] TS a3207fc803334a0a99e050b6b1fed0e8 (127.22.235.193:41869): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d39fab9c5d104e929aa10a73e6245697: Network error: Client connection negotiation failed: client connection to 127.22.235.193:41869: connect: Connection refused (error 111)
I20260808 08:26:42.845921 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 10642
I20260808 08:26:42.874178 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 10000
I20260808 08:26:42.888032 23471 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 08:26:43.232821 23471 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 08:26:43.232952 23471 mini_ranger.cc:66] Stopping Ranger...
I20260808 08:26:43.576858 23471 mini_ranger.cc:68] Stopped Ranger
2026-08-08 08:26:43.577 UTC [9282] LOG: received smart shutdown request
2026-08-08 08:26:43.580 UTC [9282] LOG: background worker "logical replication launcher" (PID 9298) exited with exit code 1
2026-08-08 08:26:43.581 UTC [9291] LOG: shutting down
2026-08-08 08:26:43.718 UTC [9282] LOG: database system is shut down
2026-08-08T08:26:43Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (45916 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 08:26:43 dist-test-slave-f7th krb5kdc[10988](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 08:26:43 dist-test-slave-f7th krb5kdc[10988](info): set up 2 sockets
Aug 08 08:26:43 dist-test-slave-f7th krb5kdc[10988](info): commencing operation
krb5kdc: starting...
W20260808 08:26:44.984534 23471 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Aug 08 08:26:45 dist-test-slave-f7th krb5kdc[10988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T08:26:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:26:45Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.22.235.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.22.235.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.22.235.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/rangeradmin_127.22.235.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.22.235.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/rangeradmin_127.22.235.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.22.235.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.22.235.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.22.235.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/rangerlookup_127.22.235.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.22.235.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/rangerlookup_127.22.235.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.22.235.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.22.235.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.22.235.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/HTTP_127.22.235.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.22.235.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/HTTP_127.22.235.212@KRBTEST.COM.keytab.
I20260808 08:26:45.153220 23471 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskGDnrCb/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/postgres -l logfile start
2026-08-08 08:26:47.361 UTC [11043] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 08:26:47.361 UTC [11043] LOG: listening on IPv4 address "127.22.235.212", port 39905
2026-08-08 08:26:47.365 UTC [11043] LOG: listening on Unix socket "/tmp/.s.PGSQL.39905"
2026-08-08 08:26:47.385 UTC [11048] LOG: database system was shut down at 2026-08-08 08:26:46 UTC
2026-08-08 08:26:47.388 UTC [11043] LOG: database system is ready to accept connections
I20260808 08:26:48.460036 23471 mini_postgres.cc:96] Postgres bound to 39905
2026-08-08 08:26:48.467 UTC [11062] FATAL: database "slave" does not exist
127.22.235.212:39905 - accepting connections
I20260808 08:26:48.467774 23471 mini_ranger.cc:161] Starting Ranger...
I20260808 08:26:48.482676 23471 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 08:26:48.885568 23471 mini_ranger.cc:87] Created ranger Postgres database
I20260808 08:26:48.885653 23471 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ranger-admin
2026-08-08 08:26:49,155 [I] DB FLAVOR :POSTGRES
2026-08-08 08:26:49,156 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 08:26:49,156 [I] Checking connection..
2026-08-08 08:26:49,156 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.235.212:39905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 08:26:49,415 [I] Checking connection passed.
2026-08-08 08:26:49,415 [I] --------- Verifying version history table ---------
2026-08-08 08:26:49,415 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.235.212:39905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 08:26:49,672 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 08:26:49,673 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.235.212:39905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 08:26:49,927 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 08:26:49,928 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 08:26:49,928 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.235.212:39905/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 08:26:50.169 UTC [11187] WARNING: there is no transaction in progress
2026-08-08 08:26:50,177 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 08:26:50,177 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.235.212:39905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 08:26:50,434 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 08:26:50,434 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 08:26:50,434 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.235.212:39905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 08:26:50,662 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.235.212:39905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 08:26:50,915 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.235.212:39905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-f7th.c.gcp-upstream.internal','N') ;"
2026-08-08 08:26:51,163 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.235.212:39905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 08:26:51,404 [I] Table x_portal_user does not exist in database ranger
2026-08-08 08:26:51,404 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.235.212:39905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 08:26:51,658 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 08:26:51,659 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.235.212:39905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 08:26:51,915 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.235.212:39905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 08:26:52,162 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 08:26:52,162 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.235.212:39905/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 08:26:53.047 UTC [11376] WARNING: there is no transaction in progress
2026-08-08 08:26:53.062 UTC [11376] WARNING: there is no transaction in progress
2026-08-08 08:26:53.078 UTC [11376] WARNING: there is no transaction in progress
2026-08-08 08:26:53.092 UTC [11376] WARNING: there is no transaction in progress
2026-08-08 08:26:53.106 UTC [11376] WARNING: there is no transaction in progress
2026-08-08 08:26:53.122 UTC [11376] WARNING: there is no transaction in progress
2026-08-08 08:26:53.172 UTC [11376] WARNING: there is no transaction in progress
2026-08-08 08:26:53.178 UTC [11376] WARNING: there is no transaction in progress
2026-08-08 08:26:53.191 UTC [11376] WARNING: there is no transaction in progress
2026-08-08 08:26:53.200 UTC [11376] WARNING: there is no transaction in progress
2026-08-08 08:26:53.209 UTC [11376] WARNING: there is no transaction in progress
2026-08-08 08:26:53.222 UTC [11376] WARNING: there is no transaction in progress
2026-08-08 08:26:53.231 UTC [11376] WARNING: there is no transaction in progress
2026-08-08 08:26:53.236 UTC [11376] WARNING: there is no transaction in progress
2026-08-08 08:26:53.242 UTC [11376] WARNING: there is no transaction in progress
2026-08-08 08:26:53.247 UTC [11376] WARNING: there is no transaction in progress
2026-08-08 08:26:53,804 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 08:26:53,804 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.235.212:39905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 08:26:54,054 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 08:26:54,054 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.235.212:39905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 08:26:54,311 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 08:26:54,311 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.235.212:39905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 08:26:54,553 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 08:26:54,554 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.235.212:39905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 08:26:54,811 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.235.212:39905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 08:26:55,067 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.235.212:39905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-f7th.c.gcp-upstream.internal';"
2026-08-08 08:26:55,316 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 08:26:55,316 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.235.212:39905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 08:26:55,571 [I] DB_PATCHES have already been applied
I20260808 08:26:55.577312 23471 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ranger-admin:/tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskGDnrCb/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 08:26:55.577378 23471 mini_ranger.cc:193] Using host: 127.22.235.212
I20260808 08:26:55.581535 23471 mini_ranger.cc:227] Ranger admin URL: http://127.22.235.212:43645
Aug 08, 2026 8:26:55 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 8:26:55 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 8:26:55 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 8:26:55 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 8:26:56 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 8:26:56 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 8:26:56 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 08:26:56.839843 23471 mini_ranger.cc:160] Time spent starting Ranger: real 8.372s user 0.002s sys 0.008s
Aug 08, 2026 8:27:17 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 8:27:17 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 08, 2026 8:27:17 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 8:27:17 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 8:27:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 8:27:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 8:27:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 8:27:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 8:27:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 8:27:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 8:27:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 8:27:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 8:27:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 8:27:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 8:27:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 8:27:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 8:27:17 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 8:27:18 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 08:27:18.873281 23471 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.22.235.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.22.235.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.22.235.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/rangerkms_127.22.235.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.22.235.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/rangerkms_127.22.235.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.22.235.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.22.235.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.22.235.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/HTTP_127.22.235.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.22.235.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/HTTP_127.22.235.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 08:27:18 dist-test-slave-f7th krb5kdc[10988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177638, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 08:27:18.962405 23471 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 08:27:18.972591 23471 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 08:27:19.692212 23471 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 08:27:19.692284 23471 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ranger-kms
I20260808 08:27:19.692301 23471 mini_ranger_kms.cc:227] Using postgres at 127.22.235.212:39905
2026-08-08 08:27:19,858 [I] DB FLAVOR :POSTGRES
2026-08-08 08:27:19,858 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 08:27:19,858 [I] Checking connection
2026-08-08 08:27:19,859 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.235.212:39905/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 08:27:20,091 [I] connection success
2026-08-08 08:27:20,091 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 08:27:20,091 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 08:27:20,091 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.235.212:39905/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 08:27:20,321 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 08:27:20,322 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 08:27:20,322 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 08:27:20,322 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.235.212:39905/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 08:27:20,600 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 08:27:21.077981 23471 mini_ranger_kms.cc:310] Created kms service
I20260808 08:27:21.238812 23471 mini_ranger_kms.cc:326] Created kudu user
I20260808 08:27:21.285039 23471 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 08:27:21 dist-test-slave-f7th krb5kdc[10988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177638, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.22.235.212@KRBTEST.COM
I20260808 08:27:21.687661 23471 mini_ranger_kms.cc:384] Added ranger policy
I20260808 08:27:21.687814 23471 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ranger-kms:/tmp/dist-test-taskGDnrCb/build/debug/bin/postgresql.jar:/tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskGDnrCb/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 08:27:21.687834 23471 mini_ranger_kms.cc:240] Using host: 127.22.235.212
I20260808 08:27:21.692456 23471 mini_ranger_kms.cc:276] Ranger KMS PID: 11714
I20260808 08:27:21.692553 23471 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.22.235.212:46555
Aug 08, 2026 8:27:22 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 8:27:22 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 8:27:22 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 8:27:22 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 8:27:23 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 8:27:23 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 08:27:24 dist-test-slave-f7th krb5kdc[10988](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786177644, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.235.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:27:24.286366 23471 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.324s user 0.014s sys 0.138s
I20260808 08:27:24.286500 23471 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 08:27:24.286572 23471 mini_ranger_kms.cc:401] 127.22.235.212:46555/kms/v1/keys
Aug 08 08:27:24 dist-test-slave-f7th krb5kdc[10988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177644, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.235.212@KRBTEST.COM for HTTP/127.22.235.212@KRBTEST.COM
Aug 08 08:27:24 dist-test-slave-f7th krb5kdc[10988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177644, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.235.212@KRBTEST.COM for HTTP/127.22.235.212@KRBTEST.COM
Aug 08 08:27:27 dist-test-slave-f7th krb5kdc[10988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.22.235.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
I20260808 08:27:27.121810 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:33647
--webserver_interface=127.22.235.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:43813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:33647
--ranger_config_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.235.212:46555/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:27:27.256518 11809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:27:27.256829 11809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:27:27.256903 11809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:27:27.260972 11809 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:27:27.261029 11809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:27:27.261056 11809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:27:27.261086 11809 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:27:27.261116 11809 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:27:27.266696 11809 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:43813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/master-0/wal
--ranger_kms_url=127.22.235.212:46555/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:33647
--ranger_config_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:33647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.235.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11809
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:27:27.267760 11809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:27:27.268862 11809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:27:27.278421 11820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:27:27.278425 11822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:27:27.278465 11819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:27:27.278698 11809 server_base.cc:1034] running on GCE node
I20260808 08:27:27.279313 11809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:27:27.280376 11809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:27:27.281535 11809 hybrid_clock.cc:648] HybridClock initialized: now 1786177647281508 us; error 42 us; skew 500 ppm
Aug 08 08:27:27 dist-test-slave-f7th krb5kdc[10988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177647, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:27:27.289158 11809 init.cc:377] Logged in from keytab as kudu/127.22.235.254@KRBTEST.COM (short username kudu)
I20260808 08:27:27.290259 11809 webserver.cc:460] Webserver started at http://127.22.235.254:34277/ using document root <none> and password file <none>
I20260808 08:27:27.290822 11809 fs_manager.cc:359] Metadata directory not provided
I20260808 08:27:27.290885 11809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:27:27.291100 11809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 08:27:27 dist-test-slave-f7th krb5kdc[10988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177647, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for HTTP/127.22.235.212@KRBTEST.COM
I20260808 08:27:27.338611 11809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "8173b9409af3426e9de1d34fe2efe984"
format_stamp: "Formatted at 2026-08-08 08:27:27 on dist-test-slave-f7th"
server_key: "911c993a84d71c2d0b094cc8e7840b00"
server_key_iv: "89b752f2599002b90b27baa999607ebd"
server_key_version: "kuduclusterkey@0"
I20260808 08:27:27.339250 11809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "8173b9409af3426e9de1d34fe2efe984"
format_stamp: "Formatted at 2026-08-08 08:27:27 on dist-test-slave-f7th"
server_key: "911c993a84d71c2d0b094cc8e7840b00"
server_key_iv: "89b752f2599002b90b27baa999607ebd"
server_key_version: "kuduclusterkey@0"
I20260808 08:27:27.343161 11809 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260808 08:27:27.357576 11834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:27.358918 11809 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 08:27:27.359061 11809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "8173b9409af3426e9de1d34fe2efe984"
format_stamp: "Formatted at 2026-08-08 08:27:27 on dist-test-slave-f7th"
server_key: "911c993a84d71c2d0b094cc8e7840b00"
server_key_iv: "89b752f2599002b90b27baa999607ebd"
server_key_version: "kuduclusterkey@0"
I20260808 08:27:27.359193 11809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:27:27.373298 11809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:27:27.376178 11809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:27:27.376350 11809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:27:27.383541 11809 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.254:33647
I20260808 08:27:27.383541 11931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.254:33647 every 8 connection(s)
I20260808 08:27:27.384563 11809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:27:27.388204 11932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:27:27.389878 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 11809
I20260808 08:27:27.389987 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:27:27.390250 23471 external_mini_cluster.cc:1428] Setting key bb36b310aefd3607212366e2cdae212a
I20260808 08:27:27.396741 11932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8173b9409af3426e9de1d34fe2efe984: Bootstrap starting.
Aug 08 08:27:27 dist-test-slave-f7th krb5kdc[10988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:27:27.399521 11932 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8173b9409af3426e9de1d34fe2efe984: Neither blocks nor log segments found. Creating new log.
I20260808 08:27:27.400373 11932 log.cc:824] T 00000000000000000000000000000000 P 8173b9409af3426e9de1d34fe2efe984: Log is configured to *not* fsync() on all Append() calls
I20260808 08:27:27.402628 11932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8173b9409af3426e9de1d34fe2efe984: No bootstrap required, opened a new log
I20260808 08:27:27.404095 11935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:27.391629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37340 (local address 127.22.235.254:33647)
0808 08:27:27.392046 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:27.392062 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:27.392098 (+ 36us) server_negotiation.cc:407] Connection header received
0808 08:27:27.392803 (+ 705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:27.392824 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:27.393164 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:27.393479 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:27:27.394261 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:27.395240 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:27.396088 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:27.396547 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:27.396728 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:27.399057 (+ 2329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:27:27.399063 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:27:27.399076 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:27:27.399103 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:27:27.401281 (+ 2178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:27.401686 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:27.401692 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:27.401699 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:27.401805 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:27.402067 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:27.402070 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:27.402072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:27.402450 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:27:27.402647 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:27.403190 (+ 543us) server_negotiation.cc:299] Negotiation successful
0808 08:27:27.403408 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":113,"threads_started":1}
I20260808 08:27:27.406635 11932 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8173b9409af3426e9de1d34fe2efe984 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8173b9409af3426e9de1d34fe2efe984" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 33647 } }
I20260808 08:27:27.406930 11932 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8173b9409af3426e9de1d34fe2efe984 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:27:27.407012 11932 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8173b9409af3426e9de1d34fe2efe984 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8173b9409af3426e9de1d34fe2efe984, State: Initialized, Role: FOLLOWER
I20260808 08:27:27.407473 11932 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8173b9409af3426e9de1d34fe2efe984 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8173b9409af3426e9de1d34fe2efe984" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 33647 } }
I20260808 08:27:27.407642 11932 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8173b9409af3426e9de1d34fe2efe984 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:27:27.407756 11932 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8173b9409af3426e9de1d34fe2efe984 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:27:27.407866 11932 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8173b9409af3426e9de1d34fe2efe984 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:27:27.409017 11932 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8173b9409af3426e9de1d34fe2efe984 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8173b9409af3426e9de1d34fe2efe984" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 33647 } }
I20260808 08:27:27.409380 11932 leader_election.cc:302] T 00000000000000000000000000000000 P 8173b9409af3426e9de1d34fe2efe984 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8173b9409af3426e9de1d34fe2efe984; no voters:
I20260808 08:27:27.409723 11932 leader_election.cc:288] T 00000000000000000000000000000000 P 8173b9409af3426e9de1d34fe2efe984 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:27:27.409811 11937 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8173b9409af3426e9de1d34fe2efe984 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:27:27.410024 11937 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8173b9409af3426e9de1d34fe2efe984 [term 1 LEADER]: Becoming Leader. State: Replica: 8173b9409af3426e9de1d34fe2efe984, State: Running, Role: LEADER
I20260808 08:27:27.410362 11937 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8173b9409af3426e9de1d34fe2efe984 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8173b9409af3426e9de1d34fe2efe984" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 33647 } }
I20260808 08:27:27.411002 11932 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8173b9409af3426e9de1d34fe2efe984 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:27:27.411849 11940 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8173b9409af3426e9de1d34fe2efe984 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8173b9409af3426e9de1d34fe2efe984. Latest consensus state: current_term: 1 leader_uuid: "8173b9409af3426e9de1d34fe2efe984" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8173b9409af3426e9de1d34fe2efe984" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 33647 } } }
I20260808 08:27:27.412047 11940 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8173b9409af3426e9de1d34fe2efe984 [sys.catalog]: This master's current role is: LEADER
I20260808 08:27:27.412178 11939 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8173b9409af3426e9de1d34fe2efe984 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8173b9409af3426e9de1d34fe2efe984" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8173b9409af3426e9de1d34fe2efe984" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 33647 } } }
I20260808 08:27:27.412274 11939 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8173b9409af3426e9de1d34fe2efe984 [sys.catalog]: This master's current role is: LEADER
I20260808 08:27:27.412604 11949 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:27:27.415531 11949 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:27:27.416096 11932 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 08:27:27.424396 11949 catalog_manager.cc:1347] Generated new cluster ID: d23ba8d6263049458e112742c4069a85
I20260808 08:27:27.424526 11949 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:27:27.434931 11949 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:27:27.435832 11949 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:27:27.441370 11949 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8173b9409af3426e9de1d34fe2efe984: Generated new TSK 0
I20260808 08:27:27.442009 11949 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.235.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab.
Entry for principal kudu/127.22.235.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab.
I20260808 08:27:28.129881 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.235.212:46555/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.193:0
--local_ip_for_outbound_sockets=127.22.235.193
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:33647
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab
--principal=kudu/127.22.235.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:43813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:27:28.296695 12007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:27:28.297071 12007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:27:28.297153 12007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:27:28.303160 12007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:27:28.303239 12007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:27:28.303335 12007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.193
I20260808 08:27:28.311051 12007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:43813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.22.235.212:46555/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab
--principal=kudu/127.22.235.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:33647
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.193
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:27:28.312449 12007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:27:28.313827 12007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:27:28.327620 12020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:27:28.327613 12027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:27:28.327829 12023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:27:28.329913 12007 server_base.cc:1034] running on GCE node
I20260808 08:27:28.330314 12007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:27:28.331001 12007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:27:28.332178 12007 hybrid_clock.cc:648] HybridClock initialized: now 1786177648332150 us; error 38 us; skew 500 ppm
Aug 08 08:27:28 dist-test-slave-f7th krb5kdc[10988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177648, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:27:28.334863 12007 init.cc:377] Logged in from keytab as kudu/127.22.235.193@KRBTEST.COM (short username kudu)
I20260808 08:27:28.335749 12007 webserver.cc:460] Webserver started at http://127.22.235.193:39777/ using document root <none> and password file <none>
I20260808 08:27:28.336292 12007 fs_manager.cc:359] Metadata directory not provided
I20260808 08:27:28.336357 12007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:27:28.336575 12007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 08:27:28 dist-test-slave-f7th krb5kdc[10988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177648, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for HTTP/127.22.235.212@KRBTEST.COM
I20260808 08:27:28.351533 12007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-0/data/instance:
uuid: "b071c7bb3bc441828434e4cfcf1380cf"
format_stamp: "Formatted at 2026-08-08 08:27:28 on dist-test-slave-f7th"
server_key: "f763a94313e5e5d1e21f0cec7e32b19a"
server_key_iv: "9257f05136797517b16f10ce7d18da0f"
server_key_version: "kuduclusterkey@0"
I20260808 08:27:28.352246 12007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance:
uuid: "b071c7bb3bc441828434e4cfcf1380cf"
format_stamp: "Formatted at 2026-08-08 08:27:28 on dist-test-slave-f7th"
server_key: "f763a94313e5e5d1e21f0cec7e32b19a"
server_key_iv: "9257f05136797517b16f10ce7d18da0f"
server_key_version: "kuduclusterkey@0"
I20260808 08:27:28.358036 12007 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 08:27:28.376566 12041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:28.378057 12007 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.000s
I20260808 08:27:28.378232 12007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-0/wal
uuid: "b071c7bb3bc441828434e4cfcf1380cf"
format_stamp: "Formatted at 2026-08-08 08:27:28 on dist-test-slave-f7th"
server_key: "f763a94313e5e5d1e21f0cec7e32b19a"
server_key_iv: "9257f05136797517b16f10ce7d18da0f"
server_key_version: "kuduclusterkey@0"
I20260808 08:27:28.378350 12007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:27:28.410094 12007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:27:28.413695 12007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:27:28.413944 12007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:27:28.414705 12007 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:27:28.415777 12007 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:27:28.415884 12007 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:28.416023 12007 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:27:28.416105 12007 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:28.428452 12007 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.193:45183
I20260808 08:27:28.429901 12007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
I20260808 08:27:28.437288 12234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.193:45183 every 8 connection(s)
I20260808 08:27:28.439101 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 12007
I20260808 08:27:28.439229 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance
I20260808 08:27:28.439546 23471 external_mini_cluster.cc:1428] Setting key dd49836939cfcffbc83526c654189bb0
WARNING: no policy specified for kudu/127.22.235.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.194@KRBTEST.COM" created.
Aug 08 08:27:28 dist-test-slave-f7th krb5kdc[10988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177648, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:27:28.470835 12241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:28.442250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:34427 (local address 127.22.235.254:33647)
0808 08:27:28.448996 (+ 6746us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:28.449001 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:28.449021 (+ 20us) server_negotiation.cc:407] Connection header received
0808 08:27:28.450320 (+ 1299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:28.450324 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:28.450382 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:28.450476 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:27:28.459886 (+ 9410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:28.460802 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:28.462691 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:28.463034 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:28.463126 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:28.466399 (+ 3273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:27:28.466403 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:27:28.466406 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:27:28.466435 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:27:28.468303 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:28.468973 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:28.468977 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:28.468979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:28.469076 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:28.469508 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:28.469512 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:28.469514 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:28.469734 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:27:28.469812 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:28.470514 (+ 702us) server_negotiation.cc:299] Negotiation successful
0808 08:27:28.470652 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6662,"thread_start_us":87,"threads_started":1}
I20260808 08:27:28.472687 12240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:28.441095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:33647 (local address 127.22.235.193:34427)
0808 08:27:28.448817 (+ 7722us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:28.448885 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:28.449959 (+ 1074us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:28.450800 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:27:28.450810 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:27:28.451177 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:27:28.451865 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:28.451886 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:28.460992 (+ 9106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:28.460995 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:28.462545 (+ 1550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:28.462560 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:28.463372 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:28.463376 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:28.463735 (+ 359us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:28.464474 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:27:28.464500 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:27:28.466198 (+ 1698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:27:28.468451 (+ 2253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:28.468457 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:28.468490 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:28.468850 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:28.469190 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:28.469193 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:28.469195 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:28.469370 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:28.469880 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:27:28.469888 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:27:28.470213 (+ 325us) client_negotiation.cc:769] Sending connection context
0808 08:27:28.471289 (+ 1076us) client_negotiation.cc:240] Negotiation successful
0808 08:27:28.471642 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7435,"thread_start_us":149,"threads_started":1}
I20260808 08:27:28.476219 12236 heartbeater.cc:347] Connected to a master server at 127.22.235.254:33647
I20260808 08:27:28.476487 12236 heartbeater.cc:464] Registering TS with master...
I20260808 08:27:28.477238 12236 heartbeater.cc:511] Master 127.22.235.254:33647 requested a full tablet report, sending...
I20260808 08:27:28.479139 11868 ts_manager.cc:194] Registered new tserver with Master: b071c7bb3bc441828434e4cfcf1380cf (127.22.235.193:45183)
I20260808 08:27:28.480602 11868 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.193@KRBTEST.COM'} at 127.22.235.193:34427
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab.
Entry for principal kudu/127.22.235.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab.
I20260808 08:27:28.495548 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.235.212:46555/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.194:0
--local_ip_for_outbound_sockets=127.22.235.194
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:33647
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab
--principal=kudu/127.22.235.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:43813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 08:27:28 dist-test-slave-f7th krb5kdc[10988](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786177648, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 08:27:28.666172 12244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:27:28.666497 12244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:27:28.666569 12244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:27:28.673108 12244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:27:28.673190 12244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:27:28.673385 12244 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.194
I20260808 08:27:28.681273 12244 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:43813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.22.235.212:46555/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab
--principal=kudu/127.22.235.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:33647
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12244
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.194
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:27:28.682703 12244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:27:28.684249 12244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:27:28.698778 12259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:27:28.698822 12257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:27:28.699077 12244 server_base.cc:1034] running on GCE node
W20260808 08:27:28.699134 12261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:27:28.699558 12244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:27:28.700321 12244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:27:28.703682 12244 hybrid_clock.cc:648] HybridClock initialized: now 1786177648703604 us; error 96 us; skew 500 ppm
Aug 08 08:27:28 dist-test-slave-f7th krb5kdc[10988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177648, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:27:28.707397 12244 init.cc:377] Logged in from keytab as kudu/127.22.235.194@KRBTEST.COM (short username kudu)
I20260808 08:27:28.708525 12244 webserver.cc:460] Webserver started at http://127.22.235.194:33027/ using document root <none> and password file <none>
I20260808 08:27:28.709281 12244 fs_manager.cc:359] Metadata directory not provided
I20260808 08:27:28.709379 12244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:27:28.709615 12244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 08:27:28 dist-test-slave-f7th krb5kdc[10988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177648, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.194@KRBTEST.COM for HTTP/127.22.235.212@KRBTEST.COM
I20260808 08:27:28.725263 12244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-1/data/instance:
uuid: "c27ffbc262db4cb085b354474936f59f"
format_stamp: "Formatted at 2026-08-08 08:27:28 on dist-test-slave-f7th"
server_key: "90ec9a2821f13b9e385b9c74a6fcf6de"
server_key_iv: "6fc05ea5059757082c81d0b29d072143"
server_key_version: "kuduclusterkey@0"
I20260808 08:27:28.726020 12244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance:
uuid: "c27ffbc262db4cb085b354474936f59f"
format_stamp: "Formatted at 2026-08-08 08:27:28 on dist-test-slave-f7th"
server_key: "90ec9a2821f13b9e385b9c74a6fcf6de"
server_key_iv: "6fc05ea5059757082c81d0b29d072143"
server_key_version: "kuduclusterkey@0"
I20260808 08:27:28.731576 12244 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 08:27:28.749951 12272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:28.751731 12244 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260808 08:27:28.751919 12244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-1/wal
uuid: "c27ffbc262db4cb085b354474936f59f"
format_stamp: "Formatted at 2026-08-08 08:27:28 on dist-test-slave-f7th"
server_key: "90ec9a2821f13b9e385b9c74a6fcf6de"
server_key_iv: "6fc05ea5059757082c81d0b29d072143"
server_key_version: "kuduclusterkey@0"
I20260808 08:27:28.752125 12244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:27:28.800853 12244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:27:28.805037 12244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:27:28.805359 12244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:27:28.806249 12244 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:27:28.807546 12244 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:27:28.807663 12244 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:28.807798 12244 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:27:28.807869 12244 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:28.819428 12244 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.194:38915
I20260808 08:27:28.819453 12477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.194:38915 every 8 connection(s)
I20260808 08:27:28.820766 12244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
I20260808 08:27:28.831017 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 12244
I20260808 08:27:28.831138 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance
I20260808 08:27:28.831430 23471 external_mini_cluster.cc:1428] Setting key bac6b0020bdb11b41271b65e8cd6dcf4
Aug 08 08:27:28 dist-test-slave-f7th krb5kdc[10988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177648, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.194@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:27:28.842458 12241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:28.824337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:59291 (local address 127.22.235.254:33647)
0808 08:27:28.824504 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:28.824509 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:28.824531 (+ 22us) server_negotiation.cc:407] Connection header received
0808 08:27:28.825069 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:28.825074 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:28.825132 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:28.825229 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:27:28.826573 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:28.827410 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:28.829188 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:28.829555 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:28.829655 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:28.838014 (+ 8359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:27:28.838019 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:27:28.838021 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:27:28.838050 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:27:28.839763 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:28.840426 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:28.840430 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:28.840432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:28.840553 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:28.841018 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:28.841021 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:28.841023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:28.841264 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:27:28.841354 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:28.842140 (+ 786us) server_negotiation.cc:299] Negotiation successful
0808 08:27:28.842281 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
WARNING: no policy specified for kudu/127.22.235.195@KRBTEST.COM; defaulting to no policy
I20260808 08:27:28.844323 12481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:28.823231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:33647 (local address 127.22.235.194:59291)
0808 08:27:28.823757 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:28.823811 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:28.824812 (+ 1001us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:28.825410 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:27:28.825418 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:27:28.825764 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:27:28.826395 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:28.826413 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:28.827608 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:28.827611 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:28.829038 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:28.829051 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:28.829876 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:28.829880 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:28.830077 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:28.831911 (+ 1834us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:27:28.831949 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:27:28.837809 (+ 5860us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:27:28.839919 (+ 2110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:28.839926 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:28.839946 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:28.840304 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:28.840698 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:28.840702 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:28.840704 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:28.840897 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:28.841431 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:27:28.841438 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:27:28.841804 (+ 366us) client_negotiation.cc:769] Sending connection context
0808 08:27:28.842747 (+ 943us) client_negotiation.cc:240] Negotiation successful
0808 08:27:28.843112 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":157,"threads_started":1}
I20260808 08:27:28.845613 12479 heartbeater.cc:347] Connected to a master server at 127.22.235.254:33647
I20260808 08:27:28.845888 12479 heartbeater.cc:464] Registering TS with master...
I20260808 08:27:28.846508 12479 heartbeater.cc:511] Master 127.22.235.254:33647 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.195@KRBTEST.COM" created.
I20260808 08:27:28.847774 11868 ts_manager.cc:194] Registered new tserver with Master: c27ffbc262db4cb085b354474936f59f (127.22.235.194:38915)
I20260808 08:27:28.848380 11868 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.194@KRBTEST.COM'} at 127.22.235.194:59291
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab.
Entry for principal kudu/127.22.235.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab.
I20260808 08:27:28.876097 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.235.212:46555/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.195:0
--local_ip_for_outbound_sockets=127.22.235.195
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:33647
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab
--principal=kudu/127.22.235.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:43813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:27:29.010771 12488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:27:29.011165 12488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:27:29.011276 12488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:27:29.017166 12488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:27:29.017294 12488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:27:29.017418 12488 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.195
I20260808 08:27:29.025246 12488 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:43813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.22.235.212:46555/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab
--principal=kudu/127.22.235.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:33647
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12488
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.195
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:27:29.026755 12488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:27:29.028187 12488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
Aug 08 08:27:29 dist-test-slave-f7th krb5kdc[10988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177648, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for HTTP/127.22.235.212@KRBTEST.COM
W20260808 08:27:29.040899 12496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:27:29.040904 12498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:27:29.041846 12493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:27:29.042699 12488 server_base.cc:1034] running on GCE node
I20260808 08:27:29.043136 12488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:27:29.043903 12488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:27:29.045058 12488 hybrid_clock.cc:648] HybridClock initialized: now 1786177649044991 us; error 83 us; skew 500 ppm
Aug 08 08:27:29 dist-test-slave-f7th krb5kdc[10988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177649, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:27:29.048799 12488 init.cc:377] Logged in from keytab as kudu/127.22.235.195@KRBTEST.COM (short username kudu)
I20260808 08:27:29.050096 12488 webserver.cc:460] Webserver started at http://127.22.235.195:45745/ using document root <none> and password file <none>
I20260808 08:27:29.050868 12488 fs_manager.cc:359] Metadata directory not provided
I20260808 08:27:29.051014 12488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:27:29.051314 12488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 08:27:29 dist-test-slave-f7th krb5kdc[10988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177649, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.195@KRBTEST.COM for HTTP/127.22.235.212@KRBTEST.COM
I20260808 08:27:29.068846 12488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-2/data/instance:
uuid: "40004c100509460ead0a51b564116b54"
format_stamp: "Formatted at 2026-08-08 08:27:29 on dist-test-slave-f7th"
server_key: "2346b5f0b673cb7d8e889079ba47ec28"
server_key_iv: "37638839bb1c2d76e47cfcdcdc1a95bd"
server_key_version: "kuduclusterkey@0"
I20260808 08:27:29.069720 12488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance:
uuid: "40004c100509460ead0a51b564116b54"
format_stamp: "Formatted at 2026-08-08 08:27:29 on dist-test-slave-f7th"
server_key: "2346b5f0b673cb7d8e889079ba47ec28"
server_key_iv: "37638839bb1c2d76e47cfcdcdc1a95bd"
server_key_version: "kuduclusterkey@0"
I20260808 08:27:29.076831 12488 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 08:27:29.096930 12506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:29.098644 12488 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260808 08:27:29.098804 12488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-2/wal
uuid: "40004c100509460ead0a51b564116b54"
format_stamp: "Formatted at 2026-08-08 08:27:29 on dist-test-slave-f7th"
server_key: "2346b5f0b673cb7d8e889079ba47ec28"
server_key_iv: "37638839bb1c2d76e47cfcdcdc1a95bd"
server_key_version: "kuduclusterkey@0"
I20260808 08:27:29.098954 12488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:27:29.132429 12488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:27:29.136008 12488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:27:29.136191 12488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:27:29.136986 12488 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:27:29.138031 12488 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:27:29.138095 12488 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:29.138151 12488 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:27:29.138185 12488 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:29.149843 12488 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.195:43775
I20260808 08:27:29.151029 12488 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
I20260808 08:27:29.155728 12685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.195:43775 every 8 connection(s)
I20260808 08:27:29.161123 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 12488
I20260808 08:27:29.161245 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance
Aug 08 08:27:29 dist-test-slave-f7th krb5kdc[10988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177649, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.195@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:27:29.161549 23471 external_mini_cluster.cc:1428] Setting key 096c9fda9c59e157a4a2ba53906dc602
I20260808 08:27:29.165872 12241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:29.153109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:38947 (local address 127.22.235.254:33647)
0808 08:27:29.153236 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:29.153240 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:29.154366 (+ 1126us) server_negotiation.cc:407] Connection header received
0808 08:27:29.155601 (+ 1235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:29.155606 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:29.155668 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:29.155728 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:27:29.157062 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:29.157852 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:29.159069 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:29.159392 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:29.159507 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:29.162021 (+ 2514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:27:29.162026 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:27:29.162028 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:27:29.162057 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:27:29.163643 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:29.164289 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:29.164293 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:29.164295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:29.164393 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:29.164753 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:29.164756 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:29.164758 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:29.164981 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:27:29.165086 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:29.165561 (+ 475us) server_negotiation.cc:299] Negotiation successful
0808 08:27:29.165691 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 08:27:29.166616 12688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:29.153613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:33647 (local address 127.22.235.195:38947)
0808 08:27:29.154202 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:29.154259 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:29.155341 (+ 1082us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:29.156118 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:27:29.156124 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:27:29.156387 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:27:29.156891 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:29.156905 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:29.157996 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:29.157999 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:29.158925 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:29.158937 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:29.159522 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:29.159525 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:29.159662 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:29.160229 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:27:29.160245 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:27:29.161789 (+ 1544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:27:29.163804 (+ 2015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:29.163809 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:29.163824 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:29.164188 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:29.164492 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:29.164494 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:29.164496 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:29.164659 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:29.165075 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:27:29.165081 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:27:29.165310 (+ 229us) client_negotiation.cc:769] Sending connection context
0808 08:27:29.165457 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 08:27:29.165704 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":158,"threads_started":1}
I20260808 08:27:29.167801 12686 heartbeater.cc:347] Connected to a master server at 127.22.235.254:33647
I20260808 08:27:29.168035 12686 heartbeater.cc:464] Registering TS with master...
I20260808 08:27:29.168509 12686 heartbeater.cc:511] Master 127.22.235.254:33647 requested a full tablet report, sending...
I20260808 08:27:29.169641 11868 ts_manager.cc:194] Registered new tserver with Master: 40004c100509460ead0a51b564116b54 (127.22.235.195:43775)
I20260808 08:27:29.170423 11868 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.195@KRBTEST.COM'} at 127.22.235.195:38947
I20260808 08:27:29.175733 23471 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 08:27:29.176483 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 12007
I20260808 08:27:29.198655 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 12244
I20260808 08:27:29.224452 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 12488
I20260808 08:27:29.239001 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 11809
I20260808 08:27:29.254771 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:33647
--webserver_interface=127.22.235.254
--webserver_port=34277
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:43813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:33647
--ranger_config_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.235.212:46555/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.22.235.212:46555/kms with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:27:29.365841 12689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:27:29.366117 12689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:27:29.366158 12689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:27:29.370151 12689 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:27:29.370204 12689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:27:29.370221 12689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:27:29.370239 12689 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:27:29.370265 12689 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:27:29.375468 12689 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:43813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.22.235.212:46555/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:33647
--ranger_config_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:33647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.235.254
--webserver_port=34277
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12689
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:27:29.376394 12689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:27:29.377382 12689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:27:29.385403 12697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:27:29.385392 12700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:27:29.385516 12698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:27:29.385493 12689 server_base.cc:1034] running on GCE node
I20260808 08:27:29.386119 12689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:27:29.387156 12689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:27:29.388306 12689 hybrid_clock.cc:648] HybridClock initialized: now 1786177649388278 us; error 43 us; skew 500 ppm
Aug 08 08:27:29 dist-test-slave-f7th krb5kdc[10988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177649, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:27:29.390843 12689 init.cc:377] Logged in from keytab as kudu/127.22.235.254@KRBTEST.COM (short username kudu)
I20260808 08:27:29.391651 12689 webserver.cc:460] Webserver started at http://127.22.235.254:34277/ using document root <none> and password file <none>
I20260808 08:27:29.392151 12689 fs_manager.cc:359] Metadata directory not provided
I20260808 08:27:29.392223 12689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 08:27:29 dist-test-slave-f7th krb5kdc[10988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177649, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for HTTP/127.22.235.212@KRBTEST.COM
I20260808 08:27:29.467896 12689 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 08:27:29.470327 12715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:29.471652 12689 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 08:27:29.471783 12689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "8173b9409af3426e9de1d34fe2efe984"
format_stamp: "Formatted at 2026-08-08 08:27:27 on dist-test-slave-f7th"
server_key: "911c993a84d71c2d0b094cc8e7840b00"
server_key_iv: "89b752f2599002b90b27baa999607ebd"
server_key_version: "kuduclusterkey@0"
I20260808 08:27:29.472317 12689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:27:29.496745 12689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:27:29.499130 12689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:27:29.499310 12689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:27:29.506384 12689 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.254:33647
I20260808 08:27:29.506403 12809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.254:33647 every 8 connection(s)
I20260808 08:27:29.507436 12689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:27:29.512004 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 12689
I20260808 08:27:29.512482 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.235.212:46555/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.193:45183
--local_ip_for_outbound_sockets=127.22.235.193
--tserver_master_addrs=127.22.235.254:33647
--webserver_port=39777
--webserver_interface=127.22.235.193
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab
--principal=kudu/127.22.235.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:43813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.22.235.212:46555/kms with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 08:27:29.512698 12810 sys_catalog.cc:263] Verifying existing consensus state
I20260808 08:27:29.514659 12810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8173b9409af3426e9de1d34fe2efe984: Bootstrap starting.
I20260808 08:27:29.530452 12810 log.cc:824] T 00000000000000000000000000000000 P 8173b9409af3426e9de1d34fe2efe984: Log is configured to *not* fsync() on all Append() calls
I20260808 08:27:29.536763 12810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8173b9409af3426e9de1d34fe2efe984: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 08:27:29.537117 12810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8173b9409af3426e9de1d34fe2efe984: Bootstrap complete.
I20260808 08:27:29.540912 12810 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8173b9409af3426e9de1d34fe2efe984 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8173b9409af3426e9de1d34fe2efe984" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 33647 } }
I20260808 08:27:29.541307 12810 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8173b9409af3426e9de1d34fe2efe984 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8173b9409af3426e9de1d34fe2efe984, State: Initialized, Role: FOLLOWER
I20260808 08:27:29.541780 12810 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8173b9409af3426e9de1d34fe2efe984 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8173b9409af3426e9de1d34fe2efe984" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 33647 } }
I20260808 08:27:29.541893 12810 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8173b9409af3426e9de1d34fe2efe984 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:27:29.541954 12810 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8173b9409af3426e9de1d34fe2efe984 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:27:29.542055 12810 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8173b9409af3426e9de1d34fe2efe984 [term 1 FOLLOWER]: Advancing to term 2
I20260808 08:27:29.543915 12810 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8173b9409af3426e9de1d34fe2efe984 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8173b9409af3426e9de1d34fe2efe984" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 33647 } }
I20260808 08:27:29.544229 12810 leader_election.cc:302] T 00000000000000000000000000000000 P 8173b9409af3426e9de1d34fe2efe984 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8173b9409af3426e9de1d34fe2efe984; no voters:
I20260808 08:27:29.544520 12810 leader_election.cc:288] T 00000000000000000000000000000000 P 8173b9409af3426e9de1d34fe2efe984 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 08:27:29.544653 12817 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8173b9409af3426e9de1d34fe2efe984 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 08:27:29.544842 12817 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8173b9409af3426e9de1d34fe2efe984 [term 2 LEADER]: Becoming Leader. State: Replica: 8173b9409af3426e9de1d34fe2efe984, State: Running, Role: LEADER
I20260808 08:27:29.545135 12817 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8173b9409af3426e9de1d34fe2efe984 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8173b9409af3426e9de1d34fe2efe984" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 33647 } }
I20260808 08:27:29.545789 12810 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8173b9409af3426e9de1d34fe2efe984 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:27:29.546494 12820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8173b9409af3426e9de1d34fe2efe984 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "8173b9409af3426e9de1d34fe2efe984" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8173b9409af3426e9de1d34fe2efe984" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 33647 } } }
I20260808 08:27:29.546808 12820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8173b9409af3426e9de1d34fe2efe984 [sys.catalog]: This master's current role is: LEADER
I20260808 08:27:29.546703 12817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8173b9409af3426e9de1d34fe2efe984 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8173b9409af3426e9de1d34fe2efe984. Latest consensus state: current_term: 2 leader_uuid: "8173b9409af3426e9de1d34fe2efe984" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8173b9409af3426e9de1d34fe2efe984" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 33647 } } }
I20260808 08:27:29.547168 12817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8173b9409af3426e9de1d34fe2efe984 [sys.catalog]: This master's current role is: LEADER
I20260808 08:27:29.547982 12830 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:27:29.548084 12810 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 08:27:29.553972 12830 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:27:29.554669 12830 catalog_manager.cc:1259] Loaded cluster ID: d23ba8d6263049458e112742c4069a85
I20260808 08:27:29.554728 12830 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:27:29.557334 12830 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:27:29.558125 12830 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8173b9409af3426e9de1d34fe2efe984: Loaded TSK: 0
I20260808 08:27:29.559461 12830 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 08:27:29.654815 12812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:27:29.655128 12812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:27:29.655190 12812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:27:29.660991 12812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:27:29.661064 12812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:27:29.661145 12812 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.193
I20260808 08:27:29.668663 12812 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:43813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.22.235.212:46555/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab
--principal=kudu/127.22.235.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.193:45183
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.235.193
--webserver_port=39777
--tserver_master_addrs=127.22.235.254:33647
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12812
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.193
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:27:29.669991 12812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:27:29.671290 12812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:27:29.683398 12881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:27:29.683527 12879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:27:29.683785 12812 server_base.cc:1034] running on GCE node
W20260808 08:27:29.683871 12874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:27:29.684533 12812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:27:29.685369 12812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:27:29.686573 12812 hybrid_clock.cc:648] HybridClock initialized: now 1786177649686551 us; error 35 us; skew 500 ppm
Aug 08 08:27:29 dist-test-slave-f7th krb5kdc[10988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177649, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:27:29.690145 12812 init.cc:377] Logged in from keytab as kudu/127.22.235.193@KRBTEST.COM (short username kudu)
I20260808 08:27:29.691284 12812 webserver.cc:460] Webserver started at http://127.22.235.193:39777/ using document root <none> and password file <none>
I20260808 08:27:29.692026 12812 fs_manager.cc:359] Metadata directory not provided
I20260808 08:27:29.692173 12812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 08:27:29 dist-test-slave-f7th krb5kdc[10988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177649, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for HTTP/127.22.235.212@KRBTEST.COM
I20260808 08:27:29.723672 12812 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 08:27:29.729455 12894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:29.731146 12812 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.000s
I20260808 08:27:29.731298 12812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-0/wal
uuid: "b071c7bb3bc441828434e4cfcf1380cf"
format_stamp: "Formatted at 2026-08-08 08:27:28 on dist-test-slave-f7th"
server_key: "f763a94313e5e5d1e21f0cec7e32b19a"
server_key_iv: "9257f05136797517b16f10ce7d18da0f"
server_key_version: "kuduclusterkey@0"
I20260808 08:27:29.732040 12812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:27:29.774338 12812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:27:29.778074 12812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:27:29.778280 12812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:27:29.778966 12812 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:27:29.780051 12812 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:27:29.780122 12812 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:29.780186 12812 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:27:29.780213 12812 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:29.791393 12812 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.193:45183
I20260808 08:27:29.791661 13085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.193:45183 every 8 connection(s)
I20260808 08:27:29.792738 12812 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
I20260808 08:27:29.794014 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 12812
I20260808 08:27:29.794523 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.235.212:46555/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.194:38915
--local_ip_for_outbound_sockets=127.22.235.194
--tserver_master_addrs=127.22.235.254:33647
--webserver_port=33027
--webserver_interface=127.22.235.194
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab
--principal=kudu/127.22.235.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:43813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.22.235.212:46555/kms with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 08:27:29 dist-test-slave-f7th krb5kdc[10988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177649, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:27:29.827757 13093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:29.798292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:33647 (local address 127.22.235.193:40837)
0808 08:27:29.798938 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:29.798993 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:29.799970 (+ 977us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:29.810236 (+ 10266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:27:29.810247 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:27:29.810616 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:27:29.811306 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:29.811330 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:29.813334 (+ 2004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:29.813337 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:29.814720 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:29.814731 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:29.815920 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:29.815925 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:29.816124 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:29.817473 (+ 1349us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:27:29.817494 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:27:29.819077 (+ 1583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:27:29.824040 (+ 4963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:29.824047 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:29.824062 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:29.824400 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:29.824979 (+ 579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:29.824982 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:29.824985 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:29.825166 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:29.825979 (+ 813us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:27:29.825985 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:27:29.826264 (+ 279us) client_negotiation.cc:769] Sending connection context
0808 08:27:29.826477 (+ 213us) client_negotiation.cc:240] Negotiation successful
0808 08:27:29.826807 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":132,"threads_started":1}
I20260808 08:27:29.829605 13091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:29.798187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:40837 (local address 127.22.235.254:33647)
0808 08:27:29.800515 (+ 2328us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:29.800538 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:29.800582 (+ 44us) server_negotiation.cc:407] Connection header received
0808 08:27:29.800908 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:29.800926 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:29.809222 (+ 8296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:29.809650 (+ 428us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:27:29.811533 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:29.813172 (+ 1639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:29.814860 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:29.815545 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:29.815697 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:29.819306 (+ 3609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:27:29.819313 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:27:29.819332 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:27:29.819365 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:27:29.823815 (+ 4450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:29.824682 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:29.824690 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:29.824696 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:29.824854 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:29.825284 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:29.825288 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:29.825290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:29.825794 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:27:29.825913 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:29.828420 (+ 2507us) server_negotiation.cc:299] Negotiation successful
0808 08:27:29.828731 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2122,"thread_start_us":133,"threads_started":1}
I20260808 08:27:29.832185 13086 heartbeater.cc:347] Connected to a master server at 127.22.235.254:33647
I20260808 08:27:29.832453 13086 heartbeater.cc:464] Registering TS with master...
I20260808 08:27:29.833096 13086 heartbeater.cc:511] Master 127.22.235.254:33647 requested a full tablet report, sending...
W20260808 08:27:29.835290 13086 heartbeater.cc:644] Failed to heartbeat to 127.22.235.254:33647 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 08:27:29.835439 13086 heartbeater.cc:464] Registering TS with master...
I20260808 08:27:29.835624 13086 heartbeater.cc:511] Master 127.22.235.254:33647 requested a full tablet report, sending...
I20260808 08:27:29.836512 13086 heartbeater.cc:464] Registering TS with master...
I20260808 08:27:29.836745 13086 heartbeater.cc:511] Master 127.22.235.254:33647 requested a full tablet report, sending...
W20260808 08:27:29.837625 13086 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 08:27:29.929885 13090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:27:29.930135 13090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:27:29.930195 13090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:27:29.936075 13090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:27:29.936167 13090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:27:29.936249 13090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.194
I20260808 08:27:29.942972 13090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:43813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.22.235.212:46555/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab
--principal=kudu/127.22.235.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.194:38915
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.235.194
--webserver_port=33027
--tserver_master_addrs=127.22.235.254:33647
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.194
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:27:29.944420 13090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:27:29.945856 13090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:27:29.960345 13100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:27:29.960369 13104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:27:29.960600 13102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:27:29.960855 13090 server_base.cc:1034] running on GCE node
I20260808 08:27:29.961297 13090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:27:29.962013 13090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:27:29.963186 13090 hybrid_clock.cc:648] HybridClock initialized: now 1786177649963164 us; error 39 us; skew 500 ppm
Aug 08 08:27:29 dist-test-slave-f7th krb5kdc[10988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177649, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:27:29.966250 13090 init.cc:377] Logged in from keytab as kudu/127.22.235.194@KRBTEST.COM (short username kudu)
I20260808 08:27:29.967273 13090 webserver.cc:460] Webserver started at http://127.22.235.194:33027/ using document root <none> and password file <none>
I20260808 08:27:29.967875 13090 fs_manager.cc:359] Metadata directory not provided
I20260808 08:27:29.967963 13090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 08:27:29 dist-test-slave-f7th krb5kdc[10988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177649, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.194@KRBTEST.COM for HTTP/127.22.235.212@KRBTEST.COM
I20260808 08:27:30.003901 13090 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.000s
I20260808 08:27:30.007308 13116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:30.008906 13090 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 08:27:30.009055 13090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-1/wal
uuid: "c27ffbc262db4cb085b354474936f59f"
format_stamp: "Formatted at 2026-08-08 08:27:28 on dist-test-slave-f7th"
server_key: "90ec9a2821f13b9e385b9c74a6fcf6de"
server_key_iv: "6fc05ea5059757082c81d0b29d072143"
server_key_version: "kuduclusterkey@0"
I20260808 08:27:30.009780 13090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:27:30.040148 13090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:27:30.043927 13090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:27:30.044154 13090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:27:30.044912 13090 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:27:30.046103 13090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:27:30.046170 13090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:30.046231 13090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:27:30.046260 13090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:30.058890 13090 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.194:38915
I20260808 08:27:30.060222 13090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
I20260808 08:27:30.065670 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 13090
I20260808 08:27:30.066165 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.235.212:46555/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.195:43775
--local_ip_for_outbound_sockets=127.22.235.195
--tserver_master_addrs=127.22.235.254:33647
--webserver_port=45745
--webserver_interface=127.22.235.195
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab
--principal=kudu/127.22.235.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:43813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.22.235.212:46555/kms with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 08:27:30.069113 13305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.194:38915 every 8 connection(s)
Aug 08 08:27:30 dist-test-slave-f7th krb5kdc[10988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177649, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.194@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:27:30.082964 13091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:30.062313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:45769 (local address 127.22.235.254:33647)
0808 08:27:30.062461 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:30.062466 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:30.064234 (+ 1768us) server_negotiation.cc:407] Connection header received
0808 08:27:30.068201 (+ 3967us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:30.068206 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:30.068275 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:30.068334 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:27:30.070179 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:30.071022 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:30.072862 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:30.073212 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:30.073293 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:30.078729 (+ 5436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:27:30.078734 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:27:30.078737 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:27:30.078769 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:27:30.080494 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:30.081157 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:30.081161 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:30.081163 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:30.081252 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:30.081626 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:30.081629 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:30.081631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:30.081864 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:27:30.081954 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:30.082621 (+ 667us) server_negotiation.cc:299] Negotiation successful
0808 08:27:30.082762 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 08:27:30.084672 13309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:30.062822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:33647 (local address 127.22.235.194:45769)
0808 08:27:30.064042 (+ 1220us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:30.064098 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:30.065181 (+ 1083us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:30.068871 (+ 3690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:27:30.068893 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:27:30.069265 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:27:30.069984 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:30.070007 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:30.071197 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:30.071200 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:30.072711 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:30.072724 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:30.073506 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:30.073510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:30.073703 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:30.075793 (+ 2090us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:27:30.075816 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:27:30.078503 (+ 2687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:27:30.080664 (+ 2161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:30.080671 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:30.080688 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:30.081041 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:30.081350 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:30.081353 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:30.081355 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:30.081522 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:30.082025 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:27:30.082032 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:27:30.082341 (+ 309us) client_negotiation.cc:769] Sending connection context
0808 08:27:30.083191 (+ 850us) client_negotiation.cc:240] Negotiation successful
0808 08:27:30.083550 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":983,"thread_start_us":124,"threads_started":1}
I20260808 08:27:30.086105 13307 heartbeater.cc:347] Connected to a master server at 127.22.235.254:33647
I20260808 08:27:30.086369 13307 heartbeater.cc:464] Registering TS with master...
I20260808 08:27:30.087026 13307 heartbeater.cc:511] Master 127.22.235.254:33647 requested a full tablet report, sending...
W20260808 08:27:30.088685 13307 heartbeater.cc:644] Failed to heartbeat to 127.22.235.254:33647 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 08:27:30.088824 13307 heartbeater.cc:464] Registering TS with master...
I20260808 08:27:30.089031 13307 heartbeater.cc:511] Master 127.22.235.254:33647 requested a full tablet report, sending...
I20260808 08:27:30.089960 13307 heartbeater.cc:464] Registering TS with master...
I20260808 08:27:30.090155 13307 heartbeater.cc:511] Master 127.22.235.254:33647 requested a full tablet report, sending...
W20260808 08:27:30.090991 13307 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 08:27:30.256042 13311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:27:30.256294 13311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:27:30.256336 13311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:27:30.261387 13311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:27:30.261515 13311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:27:30.261626 13311 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.195
I20260808 08:27:30.268769 13311 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:43813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.22.235.212:46555/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab
--principal=kudu/127.22.235.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.195:43775
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.235.195
--webserver_port=45745
--tserver_master_addrs=127.22.235.254:33647
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.195
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:27:30.270041 13311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:27:30.271260 13311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:27:30.283648 13325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:27:30.284538 13311 server_base.cc:1034] running on GCE node
W20260808 08:27:30.289203 13320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:27:30.292933 13321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:27:30.293596 13311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:27:30.294555 13311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:27:30.297911 13311 hybrid_clock.cc:648] HybridClock initialized: now 1786177650297858 us; error 63 us; skew 500 ppm
Aug 08 08:27:30 dist-test-slave-f7th krb5kdc[10988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177650, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:27:30.301823 13311 init.cc:377] Logged in from keytab as kudu/127.22.235.195@KRBTEST.COM (short username kudu)
I20260808 08:27:30.303025 13311 webserver.cc:460] Webserver started at http://127.22.235.195:45745/ using document root <none> and password file <none>
I20260808 08:27:30.303797 13311 fs_manager.cc:359] Metadata directory not provided
I20260808 08:27:30.303942 13311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 08:27:30 dist-test-slave-f7th krb5kdc[10988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177650, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.195@KRBTEST.COM for HTTP/127.22.235.212@KRBTEST.COM
I20260808 08:27:30.324394 13311 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 08:27:30.327853 13335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:30.329551 13311 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260808 08:27:30.329687 13311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-2/wal
uuid: "40004c100509460ead0a51b564116b54"
format_stamp: "Formatted at 2026-08-08 08:27:29 on dist-test-slave-f7th"
server_key: "2346b5f0b673cb7d8e889079ba47ec28"
server_key_iv: "37638839bb1c2d76e47cfcdcdc1a95bd"
server_key_version: "kuduclusterkey@0"
I20260808 08:27:30.330385 13311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:27:30.345763 13311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:27:30.349787 13311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:27:30.350041 13311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:27:30.350745 13311 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:27:30.351820 13311 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:27:30.351873 13311 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:30.351931 13311 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:27:30.351958 13311 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:30.362224 13311 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.195:43775
I20260808 08:27:30.363286 13311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
I20260808 08:27:30.364080 13539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.195:43775 every 8 connection(s)
I20260808 08:27:30.368300 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 13311
Aug 08 08:27:30 dist-test-slave-f7th krb5kdc[10988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177650, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.195@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:27:30.380025 13091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:30.365346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:56107 (local address 127.22.235.254:33647)
0808 08:27:30.365478 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:30.365482 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:30.366436 (+ 954us) server_negotiation.cc:407] Connection header received
0808 08:27:30.367455 (+ 1019us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:30.367458 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:30.367529 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:30.367604 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:27:30.369674 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:30.370453 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:30.372139 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:30.372489 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:30.372563 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:30.375602 (+ 3039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:27:30.375606 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:27:30.375608 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:27:30.375656 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:27:30.377538 (+ 1882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:30.378180 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:30.378184 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:30.378186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:30.378286 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:30.378721 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:30.378724 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:30.378726 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:30.379013 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:27:30.379129 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:30.379720 (+ 591us) server_negotiation.cc:299] Negotiation successful
0808 08:27:30.379853 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 08:27:30.381065 13545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:30.365786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:33647 (local address 127.22.235.195:56107)
0808 08:27:30.366280 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:30.366332 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:30.367234 (+ 902us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:30.367841 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:27:30.367848 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:27:30.368205 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:27:30.369483 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:30.369509 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:30.370633 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:30.370636 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:30.371986 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:30.372010 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:30.372795 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:30.372799 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:30.373009 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:30.373639 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:27:30.373661 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:27:30.375402 (+ 1741us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:27:30.377665 (+ 2263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:30.377672 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:30.377688 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:30.378052 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:30.378395 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:30.378399 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:30.378401 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:30.378596 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:30.379122 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:27:30.379129 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:27:30.379449 (+ 320us) client_negotiation.cc:769] Sending connection context
0808 08:27:30.379661 (+ 212us) client_negotiation.cc:240] Negotiation successful
0808 08:27:30.379984 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":128,"threads_started":1}
I20260808 08:27:30.381659 13549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:30.369005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37348 (local address 127.22.235.254:33647)
0808 08:27:30.369313 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:30.369323 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:30.369726 (+ 403us) server_negotiation.cc:407] Connection header received
0808 08:27:30.369776 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:30.369778 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:30.369835 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:30.369914 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:27:30.370891 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:30.371671 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:30.372920 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:30.373221 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:30.373313 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:30.375022 (+ 1709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:27:30.375027 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:27:30.375029 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:27:30.375059 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:27:30.377144 (+ 2085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:30.378646 (+ 1502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:30.378649 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:30.378651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:30.378746 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:30.379060 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:30.379063 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:30.379065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:30.379282 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:27:30.379477 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:30.380537 (+ 1060us) server_negotiation.cc:299] Negotiation successful
0808 08:27:30.381168 (+ 631us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":130,"threads_started":1}
I20260808 08:27:30.382640 13540 heartbeater.cc:347] Connected to a master server at 127.22.235.254:33647
I20260808 08:27:30.382884 13540 heartbeater.cc:464] Registering TS with master...
I20260808 08:27:30.383394 13540 heartbeater.cc:511] Master 127.22.235.254:33647 requested a full tablet report, sending...
I20260808 08:27:30.384867 12748 ts_manager.cc:194] Registered new tserver with Master: 40004c100509460ead0a51b564116b54 (127.22.235.195:43775)
I20260808 08:27:30.386018 12748 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.195@KRBTEST.COM'} at 127.22.235.195:56107
Aug 08 08:27:30 dist-test-slave-f7th krb5kdc[10988](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786177650, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:27:30.837831 13086 heartbeater.cc:464] Registering TS with master...
I20260808 08:27:30.838165 13086 heartbeater.cc:511] Master 127.22.235.254:33647 requested a full tablet report, sending...
I20260808 08:27:30.838927 12748 ts_manager.cc:194] Registered new tserver with Master: b071c7bb3bc441828434e4cfcf1380cf (127.22.235.193:45183)
I20260808 08:27:30.839548 12748 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.193@KRBTEST.COM'} at 127.22.235.193:40837
Aug 08 08:27:31 dist-test-slave-f7th krb5kdc[10988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177650, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for HTTP/127.22.235.212@KRBTEST.COM
I20260808 08:27:31.091217 13307 heartbeater.cc:464] Registering TS with master...
I20260808 08:27:31.091521 13307 heartbeater.cc:511] Master 127.22.235.254:33647 requested a full tablet report, sending...
I20260808 08:27:31.092342 12748 ts_manager.cc:194] Registered new tserver with Master: c27ffbc262db4cb085b354474936f59f (127.22.235.194:38915)
I20260808 08:27:31.093185 12748 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.194@KRBTEST.COM'} at 127.22.235.194:45769
I20260808 08:27:31.096751 23471 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 08:27:31.109103 13584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:31.098801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42240 (local address 127.22.235.254:33647)
0808 08:27:31.099085 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:31.099090 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:31.099342 (+ 252us) server_negotiation.cc:407] Connection header received
0808 08:27:31.099536 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:31.099540 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:31.099592 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:31.099652 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:27:31.100678 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:31.101458 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:31.102759 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:31.103181 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:31.103279 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:31.105099 (+ 1820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:27:31.105103 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:27:31.105106 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:27:31.105134 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:27:31.106867 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:31.107348 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:31.107352 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:31.107354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:31.107442 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:31.107739 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:31.107742 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:31.107744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:31.107951 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:27:31.108071 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:31.108791 (+ 720us) server_negotiation.cc:299] Negotiation successful
0808 08:27:31.108921 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":105,"threads_started":1}
I20260808 08:27:31.113431 12748 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42240:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 08:27:31.118443 12748 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 08:27:31.148983 13594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:31.137054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.194:38915 (local address 127.0.0.1:49554)
0808 08:27:31.139982 (+ 2928us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:31.140036 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:31.140165 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:31.141782 (+ 1617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:27:31.141789 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:27:31.141814 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:27:31.141969 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:31.141977 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:31.143676 (+ 1699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:31.143680 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:31.146033 (+ 2353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:31.146045 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:31.147571 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:31.147575 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:31.147694 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:31.147735 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 08:27:31.147842 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 08:27:31.148019 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2743,"thread_start_us":134,"threads_started":1}
I20260808 08:27:31.150038 13595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:31.138132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49554 (local address 127.22.235.194:38915)
0808 08:27:31.140447 (+ 2315us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:31.140455 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:31.140489 (+ 34us) server_negotiation.cc:407] Connection header received
0808 08:27:31.140564 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:31.140569 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:31.140875 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:31.142121 (+ 1246us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:27:31.142258 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:31.143538 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:31.146196 (+ 2658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:31.147240 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:31.147356 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:31.147505 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:31.149785 (+ 2280us) server_negotiation.cc:299] Negotiation successful
0808 08:27:31.149895 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2203,"thread_start_us":72,"threads_started":1}
I20260808 08:27:31.153646 13195 tablet_service.cc:1467] Processing CreateTablet for tablet eeb898f311c24b2abbf2170f501f8835 (DEFAULT_TABLE table=test-table [id=6908e0e090e24131a7d4c8de5ad2bc2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:27:31.155258 13195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eeb898f311c24b2abbf2170f501f8835. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:27:31.157625 13594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:31.139901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:45183 (local address 127.0.0.1:59952)
0808 08:27:31.149581 (+ 9680us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:31.149595 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:31.149660 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:31.152534 (+ 2874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:27:31.152538 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:27:31.152557 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:27:31.152660 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:31.152667 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:31.155067 (+ 2400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:31.155071 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:31.156120 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:31.156133 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:31.157231 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:31.157235 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:31.157321 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:31.157352 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 08:27:31.157404 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 08:27:31.157489 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9624,"thread_start_us":89,"threads_started":1}
I20260808 08:27:31.159636 13601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:31.140867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59952 (local address 127.22.235.193:45183)
0808 08:27:31.152091 (+ 11224us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:31.152097 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:31.152120 (+ 23us) server_negotiation.cc:407] Connection header received
0808 08:27:31.152189 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:31.152194 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:31.152369 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:31.153741 (+ 1372us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:27:31.153908 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:31.154935 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:31.156286 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:31.157089 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:31.158155 (+ 1066us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:31.158376 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:31.158433 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 08:27:31.159484 (+ 1051us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11114,"thread_start_us":126,"threads_started":1}
I20260808 08:27:31.163157 12960 tablet_service.cc:1467] Processing CreateTablet for tablet eeb898f311c24b2abbf2170f501f8835 (DEFAULT_TABLE table=test-table [id=6908e0e090e24131a7d4c8de5ad2bc2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:27:31.164597 12960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eeb898f311c24b2abbf2170f501f8835. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:27:31.167508 13597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:31.139535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.195:43775 (local address 127.0.0.1:60512)
0808 08:27:31.144777 (+ 5242us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:31.144792 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:31.144874 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:31.154320 (+ 9446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:27:31.154323 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:27:31.154344 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:27:31.154425 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:31.154431 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:31.164748 (+ 10317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:31.164753 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:31.165859 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:31.165870 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:31.167095 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:31.167099 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:31.167199 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:31.167229 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 08:27:31.167277 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 08:27:31.167353 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5176,"thread_start_us":102,"threads_started":1}
I20260808 08:27:31.168411 13600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:31.140469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60512 (local address 127.22.235.195:43775)
0808 08:27:31.151308 (+ 10839us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:31.151315 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:31.151341 (+ 26us) server_negotiation.cc:407] Connection header received
0808 08:27:31.151419 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:31.151424 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:31.151643 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:31.151793 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:27:31.160730 (+ 8937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:31.161937 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:31.166041 (+ 4104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:31.166938 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:31.167981 (+ 1043us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:31.168124 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:31.168177 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 08:27:31.168267 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10726,"thread_start_us":146,"threads_started":1}
I20260808 08:27:31.171629 13424 tablet_service.cc:1467] Processing CreateTablet for tablet eeb898f311c24b2abbf2170f501f8835 (DEFAULT_TABLE table=test-table [id=6908e0e090e24131a7d4c8de5ad2bc2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:27:31.172813 13424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eeb898f311c24b2abbf2170f501f8835. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:27:31.181036 13603 tablet_bootstrap.cc:492] T eeb898f311c24b2abbf2170f501f8835 P b071c7bb3bc441828434e4cfcf1380cf: Bootstrap starting.
I20260808 08:27:31.183544 13603 tablet_bootstrap.cc:654] T eeb898f311c24b2abbf2170f501f8835 P b071c7bb3bc441828434e4cfcf1380cf: Neither blocks nor log segments found. Creating new log.
I20260808 08:27:31.184551 13603 log.cc:824] T eeb898f311c24b2abbf2170f501f8835 P b071c7bb3bc441828434e4cfcf1380cf: Log is configured to *not* fsync() on all Append() calls
I20260808 08:27:31.187584 13603 tablet_bootstrap.cc:492] T eeb898f311c24b2abbf2170f501f8835 P b071c7bb3bc441828434e4cfcf1380cf: No bootstrap required, opened a new log
I20260808 08:27:31.187814 13603 ts_tablet_manager.cc:1397] T eeb898f311c24b2abbf2170f501f8835 P b071c7bb3bc441828434e4cfcf1380cf: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260808 08:27:31.191929 13603 raft_consensus.cc:348] T eeb898f311c24b2abbf2170f501f8835 P b071c7bb3bc441828434e4cfcf1380cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c27ffbc262db4cb085b354474936f59f" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 38915 } } peers { permanent_uuid: "40004c100509460ead0a51b564116b54" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 43775 } } peers { permanent_uuid: "b071c7bb3bc441828434e4cfcf1380cf" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45183 } }
I20260808 08:27:31.192210 13603 raft_consensus.cc:374] T eeb898f311c24b2abbf2170f501f8835 P b071c7bb3bc441828434e4cfcf1380cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:27:31.192258 13603 raft_consensus.cc:729] T eeb898f311c24b2abbf2170f501f8835 P b071c7bb3bc441828434e4cfcf1380cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b071c7bb3bc441828434e4cfcf1380cf, State: Initialized, Role: FOLLOWER
I20260808 08:27:31.193054 13602 tablet_bootstrap.cc:492] T eeb898f311c24b2abbf2170f501f8835 P c27ffbc262db4cb085b354474936f59f: Bootstrap starting.
I20260808 08:27:31.193014 13603 consensus_queue.cc:260] T eeb898f311c24b2abbf2170f501f8835 P b071c7bb3bc441828434e4cfcf1380cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c27ffbc262db4cb085b354474936f59f" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 38915 } } peers { permanent_uuid: "40004c100509460ead0a51b564116b54" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 43775 } } peers { permanent_uuid: "b071c7bb3bc441828434e4cfcf1380cf" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45183 } }
I20260808 08:27:31.194281 13603 ts_tablet_manager.cc:1428] T eeb898f311c24b2abbf2170f501f8835 P b071c7bb3bc441828434e4cfcf1380cf: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 08:27:31.194736 13086 heartbeater.cc:503] Master 127.22.235.254:33647 was elected leader, sending a full tablet report...
I20260808 08:27:31.195571 13602 tablet_bootstrap.cc:654] T eeb898f311c24b2abbf2170f501f8835 P c27ffbc262db4cb085b354474936f59f: Neither blocks nor log segments found. Creating new log.
I20260808 08:27:31.196437 13602 log.cc:824] T eeb898f311c24b2abbf2170f501f8835 P c27ffbc262db4cb085b354474936f59f: Log is configured to *not* fsync() on all Append() calls
I20260808 08:27:31.198179 13604 tablet_bootstrap.cc:492] T eeb898f311c24b2abbf2170f501f8835 P 40004c100509460ead0a51b564116b54: Bootstrap starting.
I20260808 08:27:31.200479 13604 tablet_bootstrap.cc:654] T eeb898f311c24b2abbf2170f501f8835 P 40004c100509460ead0a51b564116b54: Neither blocks nor log segments found. Creating new log.
I20260808 08:27:31.201121 13602 tablet_bootstrap.cc:492] T eeb898f311c24b2abbf2170f501f8835 P c27ffbc262db4cb085b354474936f59f: No bootstrap required, opened a new log
I20260808 08:27:31.201360 13604 log.cc:824] T eeb898f311c24b2abbf2170f501f8835 P 40004c100509460ead0a51b564116b54: Log is configured to *not* fsync() on all Append() calls
I20260808 08:27:31.201591 13602 ts_tablet_manager.cc:1397] T eeb898f311c24b2abbf2170f501f8835 P c27ffbc262db4cb085b354474936f59f: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260808 08:27:31.207396 13602 raft_consensus.cc:348] T eeb898f311c24b2abbf2170f501f8835 P c27ffbc262db4cb085b354474936f59f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c27ffbc262db4cb085b354474936f59f" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 38915 } } peers { permanent_uuid: "40004c100509460ead0a51b564116b54" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 43775 } } peers { permanent_uuid: "b071c7bb3bc441828434e4cfcf1380cf" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45183 } }
I20260808 08:27:31.207804 13602 raft_consensus.cc:374] T eeb898f311c24b2abbf2170f501f8835 P c27ffbc262db4cb085b354474936f59f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:27:31.207921 13602 raft_consensus.cc:729] T eeb898f311c24b2abbf2170f501f8835 P c27ffbc262db4cb085b354474936f59f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c27ffbc262db4cb085b354474936f59f, State: Initialized, Role: FOLLOWER
I20260808 08:27:31.208552 13602 consensus_queue.cc:260] T eeb898f311c24b2abbf2170f501f8835 P c27ffbc262db4cb085b354474936f59f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c27ffbc262db4cb085b354474936f59f" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 38915 } } peers { permanent_uuid: "40004c100509460ead0a51b564116b54" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 43775 } } peers { permanent_uuid: "b071c7bb3bc441828434e4cfcf1380cf" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45183 } }
I20260808 08:27:31.209779 13602 ts_tablet_manager.cc:1428] T eeb898f311c24b2abbf2170f501f8835 P c27ffbc262db4cb085b354474936f59f: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 08:27:31.210194 13307 heartbeater.cc:503] Master 127.22.235.254:33647 was elected leader, sending a full tablet report...
I20260808 08:27:31.210613 13604 tablet_bootstrap.cc:492] T eeb898f311c24b2abbf2170f501f8835 P 40004c100509460ead0a51b564116b54: No bootstrap required, opened a new log
I20260808 08:27:31.210821 13604 ts_tablet_manager.cc:1397] T eeb898f311c24b2abbf2170f501f8835 P 40004c100509460ead0a51b564116b54: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.000s
I20260808 08:27:31.214720 13604 raft_consensus.cc:348] T eeb898f311c24b2abbf2170f501f8835 P 40004c100509460ead0a51b564116b54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c27ffbc262db4cb085b354474936f59f" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 38915 } } peers { permanent_uuid: "40004c100509460ead0a51b564116b54" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 43775 } } peers { permanent_uuid: "b071c7bb3bc441828434e4cfcf1380cf" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45183 } }
I20260808 08:27:31.215003 13604 raft_consensus.cc:374] T eeb898f311c24b2abbf2170f501f8835 P 40004c100509460ead0a51b564116b54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:27:31.215075 13604 raft_consensus.cc:729] T eeb898f311c24b2abbf2170f501f8835 P 40004c100509460ead0a51b564116b54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40004c100509460ead0a51b564116b54, State: Initialized, Role: FOLLOWER
I20260808 08:27:31.215513 13604 consensus_queue.cc:260] T eeb898f311c24b2abbf2170f501f8835 P 40004c100509460ead0a51b564116b54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c27ffbc262db4cb085b354474936f59f" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 38915 } } peers { permanent_uuid: "40004c100509460ead0a51b564116b54" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 43775 } } peers { permanent_uuid: "b071c7bb3bc441828434e4cfcf1380cf" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45183 } }
I20260808 08:27:31.216212 13540 heartbeater.cc:503] Master 127.22.235.254:33647 was elected leader, sending a full tablet report...
I20260808 08:27:31.225312 13604 ts_tablet_manager.cc:1428] T eeb898f311c24b2abbf2170f501f8835 P 40004c100509460ead0a51b564116b54: Time spent starting tablet: real 0.014s user 0.007s sys 0.000s
W20260808 08:27:31.306418 13088 tablet.cc:2369] T eeb898f311c24b2abbf2170f501f8835 P b071c7bb3bc441828434e4cfcf1380cf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 08:27:31.324007 13308 tablet.cc:2369] T eeb898f311c24b2abbf2170f501f8835 P c27ffbc262db4cb085b354474936f59f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 08:27:31.365301 13543 tablet.cc:2369] T eeb898f311c24b2abbf2170f501f8835 P 40004c100509460ead0a51b564116b54: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 08:27:31.380251 13614 raft_consensus.cc:482] T eeb898f311c24b2abbf2170f501f8835 P c27ffbc262db4cb085b354474936f59f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 08:27:31.380475 13614 raft_consensus.cc:504] T eeb898f311c24b2abbf2170f501f8835 P c27ffbc262db4cb085b354474936f59f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c27ffbc262db4cb085b354474936f59f" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 38915 } } peers { permanent_uuid: "40004c100509460ead0a51b564116b54" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 43775 } } peers { permanent_uuid: "b071c7bb3bc441828434e4cfcf1380cf" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45183 } }
I20260808 08:27:31.381856 13614 leader_election.cc:288] T eeb898f311c24b2abbf2170f501f8835 P c27ffbc262db4cb085b354474936f59f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 40004c100509460ead0a51b564116b54 (127.22.235.195:43775), b071c7bb3bc441828434e4cfcf1380cf (127.22.235.193:45183)
I20260808 08:27:31.388561 13601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:31.382856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:44911 (local address 127.22.235.193:45183)
0808 08:27:31.382982 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:31.382986 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:31.383651 (+ 665us) server_negotiation.cc:407] Connection header received
0808 08:27:31.383845 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:31.383849 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:31.383897 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:31.383940 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:27:31.384479 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:31.385466 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:31.387176 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:31.387760 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:31.387932 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:31.387989 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:31.388343 (+ 354us) server_negotiation.cc:299] Negotiation successful
0808 08:27:31.388427 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 08:27:31.389364 13619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:31.382718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:45183 (local address 127.22.235.194:44911)
0808 08:27:31.383565 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:31.383578 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:31.383720 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:31.384064 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:27:31.384067 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:27:31.384084 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:27:31.384338 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:31.384347 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:31.385603 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:31.385606 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:31.387018 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:31.387031 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:31.388089 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:31.388092 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:31.388195 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:31.388219 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 08:27:31.389021 (+ 802us) client_negotiation.cc:240] Negotiation successful
0808 08:27:31.389106 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":770,"thread_start_us":93,"threads_started":1}
I20260808 08:27:31.390159 13008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eeb898f311c24b2abbf2170f501f8835" candidate_uuid: "c27ffbc262db4cb085b354474936f59f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b071c7bb3bc441828434e4cfcf1380cf" is_pre_election: true
I20260808 08:27:31.390581 13008 raft_consensus.cc:2454] T eeb898f311c24b2abbf2170f501f8835 P b071c7bb3bc441828434e4cfcf1380cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c27ffbc262db4cb085b354474936f59f in term 0.
I20260808 08:27:31.391243 13117 leader_election.cc:302] T eeb898f311c24b2abbf2170f501f8835 P c27ffbc262db4cb085b354474936f59f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b071c7bb3bc441828434e4cfcf1380cf, c27ffbc262db4cb085b354474936f59f; no voters:
I20260808 08:27:31.391642 13614 raft_consensus.cc:2781] T eeb898f311c24b2abbf2170f501f8835 P c27ffbc262db4cb085b354474936f59f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 08:27:31.391752 13614 raft_consensus.cc:482] T eeb898f311c24b2abbf2170f501f8835 P c27ffbc262db4cb085b354474936f59f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 08:27:31.391824 13614 raft_consensus.cc:3037] T eeb898f311c24b2abbf2170f501f8835 P c27ffbc262db4cb085b354474936f59f [term 0 FOLLOWER]: Advancing to term 1
Aug 08 08:27:31 dist-test-slave-f7th krb5kdc[10988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177650, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for HTTP/127.22.235.212@KRBTEST.COM
I20260808 08:27:31.392371 13600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:31.382418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:48953 (local address 127.22.235.195:43775)
0808 08:27:31.383042 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:31.383047 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:31.383070 (+ 23us) server_negotiation.cc:407] Connection header received
0808 08:27:31.383122 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:31.383126 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:31.383174 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:31.383261 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:27:31.383785 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:31.384507 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:31.391056 (+ 6549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:31.391631 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:31.391735 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:31.391790 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:31.392154 (+ 364us) server_negotiation.cc:299] Negotiation successful
0808 08:27:31.392241 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525}
I20260808 08:27:31.392748 13618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:31.382286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.195:43775 (local address 127.22.235.194:48953)
0808 08:27:31.382676 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:31.382693 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:31.382853 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:31.383279 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:27:31.383281 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:27:31.383302 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:27:31.383628 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:31.383638 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:31.384689 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:31.384692 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:31.390711 (+ 6019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:31.390723 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:31.391896 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:31.391899 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:31.392002 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:31.392027 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 08:27:31.392518 (+ 491us) client_negotiation.cc:240] Negotiation successful
0808 08:27:31.392596 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":146,"threads_started":1}
I20260808 08:27:31.393267 13614 raft_consensus.cc:504] T eeb898f311c24b2abbf2170f501f8835 P c27ffbc262db4cb085b354474936f59f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c27ffbc262db4cb085b354474936f59f" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 38915 } } peers { permanent_uuid: "40004c100509460ead0a51b564116b54" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 43775 } } peers { permanent_uuid: "b071c7bb3bc441828434e4cfcf1380cf" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45183 } }
I20260808 08:27:31.393450 13426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eeb898f311c24b2abbf2170f501f8835" candidate_uuid: "c27ffbc262db4cb085b354474936f59f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40004c100509460ead0a51b564116b54" is_pre_election: true
I20260808 08:27:31.393653 13614 leader_election.cc:288] T eeb898f311c24b2abbf2170f501f8835 P c27ffbc262db4cb085b354474936f59f [CANDIDATE]: Term 1 election: Requested vote from peers 40004c100509460ead0a51b564116b54 (127.22.235.195:43775), b071c7bb3bc441828434e4cfcf1380cf (127.22.235.193:45183)
I20260808 08:27:31.393756 13426 raft_consensus.cc:2454] T eeb898f311c24b2abbf2170f501f8835 P 40004c100509460ead0a51b564116b54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c27ffbc262db4cb085b354474936f59f in term 0.
I20260808 08:27:31.394860 13426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eeb898f311c24b2abbf2170f501f8835" candidate_uuid: "c27ffbc262db4cb085b354474936f59f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40004c100509460ead0a51b564116b54"
I20260808 08:27:31.394980 13426 raft_consensus.cc:3037] T eeb898f311c24b2abbf2170f501f8835 P 40004c100509460ead0a51b564116b54 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:27:31.396265 13426 raft_consensus.cc:2454] T eeb898f311c24b2abbf2170f501f8835 P 40004c100509460ead0a51b564116b54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c27ffbc262db4cb085b354474936f59f in term 1.
I20260808 08:27:31.396685 13117 leader_election.cc:302] T eeb898f311c24b2abbf2170f501f8835 P c27ffbc262db4cb085b354474936f59f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40004c100509460ead0a51b564116b54, c27ffbc262db4cb085b354474936f59f; no voters:
I20260808 08:27:31.400949 13008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eeb898f311c24b2abbf2170f501f8835" candidate_uuid: "c27ffbc262db4cb085b354474936f59f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b071c7bb3bc441828434e4cfcf1380cf"
I20260808 08:27:31.401122 13008 raft_consensus.cc:3037] T eeb898f311c24b2abbf2170f501f8835 P b071c7bb3bc441828434e4cfcf1380cf [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:27:31.402947 13008 raft_consensus.cc:2454] T eeb898f311c24b2abbf2170f501f8835 P b071c7bb3bc441828434e4cfcf1380cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c27ffbc262db4cb085b354474936f59f in term 1.
I20260808 08:27:31.403198 13614 raft_consensus.cc:2781] T eeb898f311c24b2abbf2170f501f8835 P c27ffbc262db4cb085b354474936f59f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:27:31.403486 13614 raft_consensus.cc:686] T eeb898f311c24b2abbf2170f501f8835 P c27ffbc262db4cb085b354474936f59f [term 1 LEADER]: Becoming Leader. State: Replica: c27ffbc262db4cb085b354474936f59f, State: Running, Role: LEADER
I20260808 08:27:31.403939 13614 consensus_queue.cc:237] T eeb898f311c24b2abbf2170f501f8835 P c27ffbc262db4cb085b354474936f59f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c27ffbc262db4cb085b354474936f59f" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 38915 } } peers { permanent_uuid: "40004c100509460ead0a51b564116b54" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 43775 } } peers { permanent_uuid: "b071c7bb3bc441828434e4cfcf1380cf" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45183 } }
I20260808 08:27:31.408073 12748 catalog_manager.cc:5495] T eeb898f311c24b2abbf2170f501f8835 P c27ffbc262db4cb085b354474936f59f reported cstate change: term changed from 0 to 1, leader changed from <none> to c27ffbc262db4cb085b354474936f59f (127.22.235.194). New cstate: current_term: 1 leader_uuid: "c27ffbc262db4cb085b354474936f59f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c27ffbc262db4cb085b354474936f59f" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 38915 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40004c100509460ead0a51b564116b54" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 43775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b071c7bb3bc441828434e4cfcf1380cf" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45183 } health_report { overall_health: UNKNOWN } } }
I20260808 08:27:31.443675 13595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:31.438424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49562 (local address 127.22.235.194:38915)
0808 08:27:31.438554 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:31.438559 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:31.438574 (+ 15us) server_negotiation.cc:407] Connection header received
0808 08:27:31.438674 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:31.438677 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:31.438752 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:31.438887 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:27:31.439306 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:31.440082 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:31.441408 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:31.441747 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:31.441850 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:31.442291 (+ 441us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:27:31.442892 (+ 601us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 08:27:31.443007 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:31.443433 (+ 426us) server_negotiation.cc:299] Negotiation successful
0808 08:27:31.443500 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 08:27:31.453164 13426 raft_consensus.cc:1261] T eeb898f311c24b2abbf2170f501f8835 P 40004c100509460ead0a51b564116b54 [term 1 FOLLOWER]: Refusing update from remote peer c27ffbc262db4cb085b354474936f59f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:27:31.454051 13614 consensus_queue.cc:1035] T eeb898f311c24b2abbf2170f501f8835 P c27ffbc262db4cb085b354474936f59f [LEADER]: Connected to new peer: Peer: permanent_uuid: "40004c100509460ead0a51b564116b54" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 43775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:27:31.456379 13008 raft_consensus.cc:1261] T eeb898f311c24b2abbf2170f501f8835 P b071c7bb3bc441828434e4cfcf1380cf [term 1 FOLLOWER]: Refusing update from remote peer c27ffbc262db4cb085b354474936f59f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:27:31.457727 13614 consensus_queue.cc:1035] T eeb898f311c24b2abbf2170f501f8835 P c27ffbc262db4cb085b354474936f59f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b071c7bb3bc441828434e4cfcf1380cf" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:27:31.472184 13625 mvcc.cc:205] Tried to move back new op lower bound from 7316183660339810304 to 7316183660155387904. Current Snapshot: MvccSnapshot[applied={T|T < 7316183660339810304}]
I20260808 08:27:31.477291 13628 mvcc.cc:205] Tried to move back new op lower bound from 7316183660339810304 to 7316183660155387904. Current Snapshot: MvccSnapshot[applied={T|T < 7316183660339810304}]
I20260808 08:27:31.497737 12748 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:27:31.497964 12748 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:27:31.502139 12748 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8173b9409af3426e9de1d34fe2efe984: Sending DeleteTablet for 3 replicas of tablet eeb898f311c24b2abbf2170f501f8835
I20260808 08:27:31.503512 13424 tablet_service.cc:1514] Processing DeleteTablet for tablet eeb898f311c24b2abbf2170f501f8835 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:27:31 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:60512
I20260808 08:27:31.504155 12960 tablet_service.cc:1514] Processing DeleteTablet for tablet eeb898f311c24b2abbf2170f501f8835 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:27:31 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:59952
I20260808 08:27:31.505476 13195 tablet_service.cc:1514] Processing DeleteTablet for tablet eeb898f311c24b2abbf2170f501f8835 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:27:31 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:49554
I20260808 08:27:31.506297 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 12812
I20260808 08:27:31.506349 13642 tablet_replica.cc:330] T eeb898f311c24b2abbf2170f501f8835 P c27ffbc262db4cb085b354474936f59f: stopping tablet replica
I20260808 08:27:31.509473 13642 raft_consensus.cc:2229] T eeb898f311c24b2abbf2170f501f8835 P c27ffbc262db4cb085b354474936f59f [term 1 LEADER]: Raft consensus shutting down.
I20260808 08:27:31.510018 13642 raft_consensus.cc:2258] T eeb898f311c24b2abbf2170f501f8835 P c27ffbc262db4cb085b354474936f59f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:27:31.511659 13642 ts_tablet_manager.cc:1905] T eeb898f311c24b2abbf2170f501f8835 P c27ffbc262db4cb085b354474936f59f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:27:31.513003 13638 tablet_replica.cc:330] T eeb898f311c24b2abbf2170f501f8835 P 40004c100509460ead0a51b564116b54: stopping tablet replica
I20260808 08:27:31.513418 13638 raft_consensus.cc:2229] T eeb898f311c24b2abbf2170f501f8835 P 40004c100509460ead0a51b564116b54 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 08:27:31.513846 13638 raft_consensus.cc:2258] T eeb898f311c24b2abbf2170f501f8835 P 40004c100509460ead0a51b564116b54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:27:31.516558 13638 ts_tablet_manager.cc:1905] T eeb898f311c24b2abbf2170f501f8835 P 40004c100509460ead0a51b564116b54: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:27:31.518862 13642 ts_tablet_manager.cc:1918] T eeb898f311c24b2abbf2170f501f8835 P c27ffbc262db4cb085b354474936f59f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 08:27:31.519073 13642 log.cc:1196] T eeb898f311c24b2abbf2170f501f8835 P c27ffbc262db4cb085b354474936f59f: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-1/wal/wals/eeb898f311c24b2abbf2170f501f8835
I20260808 08:27:31.519762 13642 ts_tablet_manager.cc:1939] T eeb898f311c24b2abbf2170f501f8835 P c27ffbc262db4cb085b354474936f59f: Deleting consensus metadata
I20260808 08:27:31.519815 13638 ts_tablet_manager.cc:1918] T eeb898f311c24b2abbf2170f501f8835 P 40004c100509460ead0a51b564116b54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 08:27:31.520710 13638 log.cc:1196] T eeb898f311c24b2abbf2170f501f8835 P 40004c100509460ead0a51b564116b54: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786177493204855-23471-0/minicluster-data/ts-2/wal/wals/eeb898f311c24b2abbf2170f501f8835
I20260808 08:27:31.521224 13638 ts_tablet_manager.cc:1939] T eeb898f311c24b2abbf2170f501f8835 P 40004c100509460ead0a51b564116b54: Deleting consensus metadata
I20260808 08:27:31.521730 12720 catalog_manager.cc:4841] TS c27ffbc262db4cb085b354474936f59f (127.22.235.194:38915): tablet eeb898f311c24b2abbf2170f501f8835 (table test-table [id=6908e0e090e24131a7d4c8de5ad2bc2f]) successfully deleted
I20260808 08:27:31.523332 12718 catalog_manager.cc:4841] TS 40004c100509460ead0a51b564116b54 (127.22.235.195:43775): tablet eeb898f311c24b2abbf2170f501f8835 (table test-table [id=6908e0e090e24131a7d4c8de5ad2bc2f]) successfully deleted
W20260808 08:27:31.550882 12718 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.235.193:45183 (error 108)
I20260808 08:27:31.552898 13597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:31.552432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:45183 (local address 127.0.0.1:59958)
0808 08:27:31.552711 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:31.552798 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.193:45183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150}
W20260808 08:27:31.553239 12718 catalog_manager.cc:4584] TS b071c7bb3bc441828434e4cfcf1380cf (127.22.235.193:45183): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet eeb898f311c24b2abbf2170f501f8835: Network error: Client connection negotiation failed: client connection to 127.22.235.193:45183: connect: Connection refused (error 111)
I20260808 08:27:31.553912 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 13090
I20260808 08:27:31.566212 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 13311
I20260808 08:27:31.594017 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 12689
I20260808 08:27:31.610672 23471 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 08:27:31.956482 23471 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 08:27:31.956604 23471 mini_ranger.cc:66] Stopping Ranger...
I20260808 08:27:32.300879 23471 mini_ranger.cc:68] Stopped Ranger
2026-08-08 08:27:32.301 UTC [11043] LOG: received smart shutdown request
2026-08-08 08:27:32.305 UTC [11043] LOG: background worker "logical replication launcher" (PID 11058) exited with exit code 1
2026-08-08 08:27:32.305 UTC [11049] LOG: shutting down
2026-08-08 08:27:32.462 UTC [11043] LOG: database system is shut down
2026-08-08T08:27:32Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (48745 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 08:27:32 dist-test-slave-f7th krb5kdc[13657](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 08:27:32 dist-test-slave-f7th krb5kdc[13657](info): set up 2 sockets
Aug 08 08:27:32 dist-test-slave-f7th krb5kdc[13657](info): commencing operation
krb5kdc: starting...
W20260808 08:27:33.742307 23471 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Aug 08 08:27:33 dist-test-slave-f7th krb5kdc[13657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T08:27:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:27:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.235.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
I20260808 08:27:33.861030 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:38449
--webserver_interface=127.22.235.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:44205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:38449,127.22.235.253:46733,127.22.235.252:34823
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:27:33.971740 13682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:27:33.971999 13682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:27:33.972055 13682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:27:33.975947 13682 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:27:33.976001 13682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:27:33.976025 13682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:27:33.976047 13682 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:27:33.976068 13682 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:27:33.981175 13682 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:44205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:38449,127.22.235.253:46733,127.22.235.252:34823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:38449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.235.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13682
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:27:33.982168 13682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:27:33.983152 13682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:27:33.991387 13690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:27:33.991389 13695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:27:33.991389 13691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:27:33.991611 13682 server_base.cc:1034] running on GCE node
I20260808 08:27:33.992202 13682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:27:33.993309 13682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:27:33.994498 13682 hybrid_clock.cc:648] HybridClock initialized: now 1786177653994482 us; error 32 us; skew 500 ppm
Aug 08 08:27:33 dist-test-slave-f7th krb5kdc[13657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177653, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:27:33.997269 13682 init.cc:377] Logged in from keytab as kudu/127.22.235.254@KRBTEST.COM (short username kudu)
I20260808 08:27:33.998171 13682 webserver.cc:460] Webserver started at http://127.22.235.254:32911/ using document root <none> and password file <none>
I20260808 08:27:33.998701 13682 fs_manager.cc:359] Metadata directory not provided
I20260808 08:27:33.998760 13682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:27:33.998946 13682 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:27:34.001056 13682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "8e5c7889d202401daf1fb59c21742123"
format_stamp: "Formatted at 2026-08-08 08:27:33 on dist-test-slave-f7th"
server_key: "f521f0d7b24fe92af6a3d29fa327f8ce"
server_key_iv: "04d3349814eb32e2c8698f37b147fc2f"
server_key_version: "encryptionkey@0"
I20260808 08:27:34.001561 13682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "8e5c7889d202401daf1fb59c21742123"
format_stamp: "Formatted at 2026-08-08 08:27:33 on dist-test-slave-f7th"
server_key: "f521f0d7b24fe92af6a3d29fa327f8ce"
server_key_iv: "04d3349814eb32e2c8698f37b147fc2f"
server_key_version: "encryptionkey@0"
I20260808 08:27:34.005309 13682 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 08:27:34.007717 13706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:34.009020 13682 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:27:34.009132 13682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "8e5c7889d202401daf1fb59c21742123"
format_stamp: "Formatted at 2026-08-08 08:27:33 on dist-test-slave-f7th"
server_key: "f521f0d7b24fe92af6a3d29fa327f8ce"
server_key_iv: "04d3349814eb32e2c8698f37b147fc2f"
server_key_version: "encryptionkey@0"
I20260808 08:27:34.009231 13682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:27:34.040205 13682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:27:34.043430 13682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:27:34.043605 13682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:27:34.050371 13682 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.254:38449
I20260808 08:27:34.050631 13790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.254:38449 every 8 connection(s)
I20260808 08:27:34.051301 13682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:27:34.054111 13791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:27:34.057047 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 13682
I20260808 08:27:34.057171 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:27:34.057469 23471 external_mini_cluster.cc:1428] Setting key df0bdafd9865c300dc89f8b5890dd2e4
I20260808 08:27:34.058161 13791 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38449 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.22.235.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.253@KRBTEST.COM" created.
Aug 08 08:27:34 dist-test-slave-f7th krb5kdc[13657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177653, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:27:34.073735 13798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:34.060737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56702 (local address 127.22.235.254:38449)
0808 08:27:34.061131 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:34.061151 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:34.061454 (+ 303us) server_negotiation.cc:407] Connection header received
0808 08:27:34.062419 (+ 965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:34.062424 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:34.062610 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:34.062774 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:27:34.063594 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:34.064442 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:34.065490 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:34.065838 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:34.065984 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:34.069241 (+ 3257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:27:34.069249 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:27:34.069262 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:27:34.069291 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:27:34.070840 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:34.071349 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:34.071353 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:34.071354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:34.071453 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:34.071776 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:34.071778 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:34.071779 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:34.072130 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:27:34.072267 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:34.072730 (+ 463us) server_negotiation.cc:299] Negotiation successful
0808 08:27:34.072975 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":264,"thread_start_us":107,"threads_started":1}
I20260808 08:27:34.073738 13799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:34.060738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:38449 (local address 127.0.0.1:56702)
0808 08:27:34.061321 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:34.061353 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:34.062127 (+ 774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:34.062787 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:27:34.062792 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:27:34.063222 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:27:34.063454 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:34.063467 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:34.064566 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:34.064569 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:34.065369 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:34.065379 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:34.065988 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:34.065993 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:34.066100 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:34.066603 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:27:34.066620 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:27:34.069084 (+ 2464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:27:34.070948 (+ 1864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:34.070957 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:34.070966 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:34.071251 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:34.071551 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:34.071554 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:34.071556 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:34.071695 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:34.072259 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:27:34.072266 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:27:34.072434 (+ 168us) client_negotiation.cc:769] Sending connection context
0808 08:27:34.072644 (+ 210us) client_negotiation.cc:240] Negotiation successful
0808 08:27:34.073053 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"mutex_wait_us":86,"spinlock_wait_cycles":17152,"thread_start_us":156,"threads_started":1}
I20260808 08:27:34.075927 13791 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46733 } has no permanent_uuid. Determining permanent_uuid...
I20260808 08:27:34.076583 13799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:34.076352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.253:46733 (local address 127.0.0.1:50694)
0808 08:27:34.076459 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:34.076529 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.253:46733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 08:27:34.076759 13710 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.235.253:46733: connect: Connection refused (error 111)
I20260808 08:27:34.077531 13799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:34.077327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.253:46733 (local address 127.0.0.1:50702)
0808 08:27:34.077436 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:34.077481 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.253:46733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 08:27:34.077661 13791 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.235.253:46733: Network error: Client connection negotiation failed: client connection to 127.22.235.253:46733: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.253.keytab.
Entry for principal kudu/127.22.235.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.253.keytab.
I20260808 08:27:34.083156 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.253:46733
--webserver_interface=127.22.235.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.253.keytab
--principal=kudu/127.22.235.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:44205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:38449,127.22.235.253:46733,127.22.235.252:34823
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 08:27:34.126881 13791 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46733 } attempt: 1
I20260808 08:27:34.127754 13799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:34.127452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.253:46733 (local address 127.0.0.1:50704)
0808 08:27:34.127606 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:34.127671 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.253:46733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 08:27:34.128453 13799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:34.128236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.253:46733 (local address 127.0.0.1:50706)
0808 08:27:34.128344 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:34.128389 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.253:46733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 08:27:34.128638 13791 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.235.253:46733: Network error: Client connection negotiation failed: client connection to 127.22.235.253:46733: connect: Connection refused (error 111)
W20260808 08:27:34.193329 13802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:27:34.193569 13802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:27:34.193625 13802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 08:27:34.196786 13791 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46733 } attempt: 2
I20260808 08:27:34.197587 13799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:34.197298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.253:46733 (local address 127.0.0.1:50712)
0808 08:27:34.197442 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:34.197505 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.253:46733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 08:27:34.197651 13802 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:27:34.197726 13802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:27:34.197762 13802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:27:34.197816 13802 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:27:34.197849 13802 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:27:34.198293 13799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:34.198075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.253:46733 (local address 127.0.0.1:50728)
0808 08:27:34.198185 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:34.198233 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.253:46733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 08:27:34.198428 13791 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.235.253:46733: Network error: Client connection negotiation failed: client connection to 127.22.235.253:46733: connect: Connection refused (error 111)
I20260808 08:27:34.203018 13802 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:44205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:38449,127.22.235.253:46733,127.22.235.252:34823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.253.keytab
--principal=kudu/127.22.235.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.253:46733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.235.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:27:34.203982 13802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:27:34.204981 13802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:27:34.212896 13814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:27:34.212855 13813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:27:34.212829 13817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:27:34.213095 13802 server_base.cc:1034] running on GCE node
I20260808 08:27:34.213681 13802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:27:34.214712 13802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:27:34.215865 13802 hybrid_clock.cc:648] HybridClock initialized: now 1786177654215847 us; error 27 us; skew 500 ppm
Aug 08 08:27:34 dist-test-slave-f7th krb5kdc[13657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177654, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:27:34.218461 13802 init.cc:377] Logged in from keytab as kudu/127.22.235.253@KRBTEST.COM (short username kudu)
I20260808 08:27:34.219254 13802 webserver.cc:460] Webserver started at http://127.22.235.253:38571/ using document root <none> and password file <none>
I20260808 08:27:34.219750 13802 fs_manager.cc:359] Metadata directory not provided
I20260808 08:27:34.219805 13802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:27:34.219966 13802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:27:34.221886 13802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-1/data/instance:
uuid: "086b16b15ca94487b033ccdb71d73582"
format_stamp: "Formatted at 2026-08-08 08:27:34 on dist-test-slave-f7th"
server_key: "2dbee67fc322b6ed0c8097feeb1cd613"
server_key_iv: "bd36aa871c9ad4e8e9a867fcae8663c9"
server_key_version: "encryptionkey@0"
I20260808 08:27:34.222368 13802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-1/wal/instance:
uuid: "086b16b15ca94487b033ccdb71d73582"
format_stamp: "Formatted at 2026-08-08 08:27:34 on dist-test-slave-f7th"
server_key: "2dbee67fc322b6ed0c8097feeb1cd613"
server_key_iv: "bd36aa871c9ad4e8e9a867fcae8663c9"
server_key_version: "encryptionkey@0"
I20260808 08:27:34.225708 13802 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 08:27:34.227902 13824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:34.229101 13802 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 08:27:34.229204 13802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-1/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-1/wal
uuid: "086b16b15ca94487b033ccdb71d73582"
format_stamp: "Formatted at 2026-08-08 08:27:34 on dist-test-slave-f7th"
server_key: "2dbee67fc322b6ed0c8097feeb1cd613"
server_key_iv: "bd36aa871c9ad4e8e9a867fcae8663c9"
server_key_version: "encryptionkey@0"
I20260808 08:27:34.229293 13802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:27:34.242782 13802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:27:34.245764 13802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:27:34.245939 13802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:27:34.252264 13802 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.253:46733
I20260808 08:27:34.252514 13914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.253:46733 every 8 connection(s)
I20260808 08:27:34.253191 13802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-1/data/info.pb
I20260808 08:27:34.255990 13915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:27:34.259019 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 13802
I20260808 08:27:34.259145 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-1/wal/instance
I20260808 08:27:34.259431 23471 external_mini_cluster.cc:1428] Setting key 0794cc55e9089cc726aabdd4c136fc39
I20260808 08:27:34.260136 13915 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38449 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.22.235.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.252@KRBTEST.COM" created.
Aug 08 08:27:34 dist-test-slave-f7th krb5kdc[13657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177654, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.253@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:27:34.275280 13798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:34.262420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56706 (local address 127.22.235.254:38449)
0808 08:27:34.262545 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:34.262549 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:34.263613 (+ 1064us) server_negotiation.cc:407] Connection header received
0808 08:27:34.264522 (+ 909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:34.264525 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:34.264590 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:34.264723 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:27:34.265883 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:34.266399 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:34.267520 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:34.267760 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:34.267853 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:34.271694 (+ 3841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:27:34.271698 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:27:34.271701 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:27:34.271733 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:27:34.273132 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:34.273737 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:34.273741 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:34.273742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:34.273825 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:34.274146 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:34.274150 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:34.274151 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:34.274341 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:27:34.274434 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:34.274974 (+ 540us) server_negotiation.cc:299] Negotiation successful
0808 08:27:34.275112 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 08:27:34.276069 13923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:34.262916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:38449 (local address 127.0.0.1:56706)
0808 08:27:34.263458 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:34.263501 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:34.264300 (+ 799us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:34.264889 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:27:34.264896 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:27:34.265240 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:27:34.265728 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:34.265741 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:34.266527 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:34.266530 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:34.267393 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:34.267403 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:34.267904 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:34.267908 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:34.268073 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:34.268638 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:27:34.268661 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:27:34.271519 (+ 2858us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:27:34.273249 (+ 1730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:34.273255 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:34.273271 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:34.273634 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:34.273911 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:34.273915 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:34.273917 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:34.274055 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:34.274447 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:27:34.274455 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:27:34.274717 (+ 262us) client_negotiation.cc:769] Sending connection context
0808 08:27:34.274889 (+ 172us) client_negotiation.cc:240] Negotiation successful
0808 08:27:34.275214 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":156,"threads_started":1}
I20260808 08:27:34.277479 13915 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46733 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 08:27:34 dist-test-slave-f7th krb5kdc[13657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177654, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.253@KRBTEST.COM for kudu/127.22.235.253@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.252.keytab.
Entry for principal kudu/127.22.235.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.252.keytab.
I20260808 08:27:34.285778 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.252:34823
--webserver_interface=127.22.235.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.252.keytab
--principal=kudu/127.22.235.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:44205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:38449,127.22.235.253:46733,127.22.235.252:34823
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 08:27:34.286222 13923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:34.278012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.253:46733 (local address 127.0.0.1:50736)
0808 08:27:34.278195 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:34.278209 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:34.278288 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:34.278724 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:27:34.278727 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:27:34.278884 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:27:34.279003 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:34.279009 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:34.279908 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:34.279910 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:34.280443 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:34.280451 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:34.281097 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:34.281100 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:34.281177 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:34.281596 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:27:34.281607 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:27:34.282742 (+ 1135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:27:34.284859 (+ 2117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:34.284862 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:34.284863 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:34.285064 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:34.285324 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:34.285326 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:34.285329 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:34.285398 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:34.285805 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:27:34.285808 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:27:34.285892 (+ 84us) client_negotiation.cc:769] Sending connection context
0808 08:27:34.285990 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 08:27:34.286104 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111}
I20260808 08:27:34.286718 13925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:34.278090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50736 (local address 127.22.235.253:46733)
0808 08:27:34.278382 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:34.278391 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:34.278410 (+ 19us) server_negotiation.cc:407] Connection header received
0808 08:27:34.278445 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:34.278449 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:34.278596 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:34.278699 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:27:34.279094 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:34.279825 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:34.280548 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:34.280976 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:34.281062 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:34.282889 (+ 1827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:27:34.282899 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:27:34.282911 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:27:34.282933 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:27:34.284759 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:34.285161 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:34.285167 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:34.285169 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:34.285245 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:34.285471 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:34.285474 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:34.285476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:34.285711 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:27:34.285830 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:34.286450 (+ 620us) server_negotiation.cc:299] Negotiation successful
0808 08:27:34.286599 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":118,"threads_started":1}
I20260808 08:27:34.288703 13915 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.235.252" port: 34823 } has no permanent_uuid. Determining permanent_uuid...
I20260808 08:27:34.289361 13923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:34.289141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:34823 (local address 127.0.0.1:52382)
0808 08:27:34.289241 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:34.289307 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.252:34823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 08:27:34.289569 13825 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.235.252:34823: connect: Connection refused (error 111)
I20260808 08:27:34.289584 13791 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46733 } attempt: 3
I20260808 08:27:34.290479 13923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:34.290259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:34823 (local address 127.0.0.1:52386)
0808 08:27:34.290359 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:34.290404 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.252:34823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 08:27:34.290690 13915 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.235.252:34823: Network error: Client connection negotiation failed: client connection to 127.22.235.252:34823: connect: Connection refused (error 111)
Aug 08 08:27:34 dist-test-slave-f7th krb5kdc[13657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177653, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for kudu/127.22.235.253@KRBTEST.COM
I20260808 08:27:34.297940 13925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:34.290199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50740 (local address 127.22.235.253:46733)
0808 08:27:34.290279 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:34.290283 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:34.290308 (+ 25us) server_negotiation.cc:407] Connection header received
0808 08:27:34.290409 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:34.290412 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:34.290450 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:34.290527 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:27:34.291093 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:34.291603 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:34.292364 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:34.292711 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:34.292801 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:34.295136 (+ 2335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:27:34.295139 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:27:34.295142 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:27:34.295167 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:27:34.296455 (+ 1288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:34.296879 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:34.296883 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:34.296884 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:34.296956 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:34.297203 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:34.297206 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:34.297208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:34.297371 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:27:34.297453 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:34.297703 (+ 250us) server_negotiation.cc:299] Negotiation successful
0808 08:27:34.297803 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 08:27:34.297941 13799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:34.289984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.253:46733 (local address 127.0.0.1:50740)
0808 08:27:34.290250 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:34.290263 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:34.290338 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:34.290641 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:27:34.290645 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:27:34.290827 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:27:34.290942 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:34.290949 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:34.291709 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:34.291712 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:34.292261 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:34.292270 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:34.292809 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:34.292812 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:34.292920 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:34.293369 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:27:34.293378 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:27:34.295017 (+ 1639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:27:34.296555 (+ 1538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:34.296558 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:34.296560 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:34.296796 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:34.297039 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:34.297042 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:34.297045 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:34.297126 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:34.297456 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:27:34.297459 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:27:34.297561 (+ 102us) client_negotiation.cc:769] Sending connection context
0808 08:27:34.297645 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 08:27:34.297781 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175}
I20260808 08:27:34.299260 13791 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.235.252" port: 34823 } has no permanent_uuid. Determining permanent_uuid...
I20260808 08:27:34.299875 13799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:34.299670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:34823 (local address 127.0.0.1:52398)
0808 08:27:34.299773 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:34.299822 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.252:34823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 08:27:34.300540 13799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:34.300307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:34823 (local address 127.0.0.1:52414)
0808 08:27:34.300442 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:34.300484 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.252:34823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260808 08:27:34.300670 13791 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.235.252:34823: Network error: Client connection negotiation failed: client connection to 127.22.235.252:34823: connect: Connection refused (error 111)
I20260808 08:27:34.331816 13791 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.235.252" port: 34823 } attempt: 1
I20260808 08:27:34.332707 13799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:34.332356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:34823 (local address 127.0.0.1:52422)
0808 08:27:34.332508 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:34.332593 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.252:34823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 08:27:34.333441 13799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:34.333192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:34823 (local address 127.0.0.1:52432)
0808 08:27:34.333318 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:34.333372 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.252:34823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 08:27:34.333570 13791 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.235.252:34823: Network error: Client connection negotiation failed: client connection to 127.22.235.252:34823: connect: Connection refused (error 111)
I20260808 08:27:34.339929 13915 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.235.252" port: 34823 } attempt: 1
I20260808 08:27:34.340598 13923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:34.340359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:34823 (local address 127.0.0.1:52446)
0808 08:27:34.340480 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:34.340535 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.252:34823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 08:27:34.341279 13923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:34.341082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:34823 (local address 127.0.0.1:52450)
0808 08:27:34.341170 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:34.341222 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.252:34823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 08:27:34.341390 13915 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.235.252:34823: Network error: Client connection negotiation failed: client connection to 127.22.235.252:34823: connect: Connection refused (error 111)
W20260808 08:27:34.400408 13926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:27:34.400687 13926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:27:34.400748 13926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:27:34.404678 13926 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:27:34.404731 13926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:27:34.404757 13926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:27:34.404778 13926 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:27:34.404804 13926 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:27:34.408674 13791 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.235.252" port: 34823 } attempt: 2
I20260808 08:27:34.409404 13799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:34.409129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:34823 (local address 127.0.0.1:52466)
0808 08:27:34.409268 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:34.409328 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.252:34823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 08:27:34.409564 13915 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.235.252" port: 34823 } attempt: 2
I20260808 08:27:34.410198 13799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:34.409960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:34823 (local address 127.0.0.1:52474)
0808 08:27:34.410104 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:34.410145 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.252:34823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260808 08:27:34.410220 13923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:34.409936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:34823 (local address 127.0.0.1:52468)
0808 08:27:34.410075 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:34.410139 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.252:34823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 08:27:34.410251 13926 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:44205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:38449,127.22.235.253:46733,127.22.235.252:34823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.252.keytab
--principal=kudu/127.22.235.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.252:34823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.235.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
W20260808 08:27:34.410359 13791 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.235.252:34823: Network error: Client connection negotiation failed: client connection to 127.22.235.252:34823: connect: Connection refused (error 111)
I20260808 08:27:34.410909 13923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:34.410716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:34823 (local address 127.0.0.1:52484)
0808 08:27:34.410814 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:34.410857 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.252:34823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 08:27:34.411026 13915 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.235.252:34823: Network error: Client connection negotiation failed: client connection to 127.22.235.252:34823: connect: Connection refused (error 111)
I20260808 08:27:34.411356 13926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:27:34.412323 13926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:27:34.420189 13937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:27:34.420205 13938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:27:34.420228 13940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:27:34.420406 13926 server_base.cc:1034] running on GCE node
I20260808 08:27:34.421097 13926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:27:34.422154 13926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:27:34.423318 13926 hybrid_clock.cc:648] HybridClock initialized: now 1786177654423297 us; error 30 us; skew 500 ppm
Aug 08 08:27:34 dist-test-slave-f7th krb5kdc[13657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177654, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:27:34.425897 13926 init.cc:377] Logged in from keytab as kudu/127.22.235.252@KRBTEST.COM (short username kudu)
I20260808 08:27:34.426708 13926 webserver.cc:460] Webserver started at http://127.22.235.252:40693/ using document root <none> and password file <none>
I20260808 08:27:34.427241 13926 fs_manager.cc:359] Metadata directory not provided
I20260808 08:27:34.427306 13926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:27:34.427510 13926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:27:34.429518 13926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-2/data/instance:
uuid: "04f4161cd07846e4a4bf5ef1c6eb0b04"
format_stamp: "Formatted at 2026-08-08 08:27:34 on dist-test-slave-f7th"
server_key: "7db2f50ab2663bfb3fc417ba865682d8"
server_key_iv: "912822c27ff5da2b886c3cae5066f652"
server_key_version: "encryptionkey@0"
I20260808 08:27:34.430047 13926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-2/wal/instance:
uuid: "04f4161cd07846e4a4bf5ef1c6eb0b04"
format_stamp: "Formatted at 2026-08-08 08:27:34 on dist-test-slave-f7th"
server_key: "7db2f50ab2663bfb3fc417ba865682d8"
server_key_iv: "912822c27ff5da2b886c3cae5066f652"
server_key_version: "encryptionkey@0"
I20260808 08:27:34.433629 13926 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 08:27:34.436003 13959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:34.437278 13926 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260808 08:27:34.437391 13926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-2/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-2/wal
uuid: "04f4161cd07846e4a4bf5ef1c6eb0b04"
format_stamp: "Formatted at 2026-08-08 08:27:34 on dist-test-slave-f7th"
server_key: "7db2f50ab2663bfb3fc417ba865682d8"
server_key_iv: "912822c27ff5da2b886c3cae5066f652"
server_key_version: "encryptionkey@0"
I20260808 08:27:34.437480 13926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:27:34.458549 13926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:27:34.461319 13926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:27:34.461490 13926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:27:34.468155 13926 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.252:34823
I20260808 08:27:34.468425 14054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.252:34823 every 8 connection(s)
I20260808 08:27:34.469110 13926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-2/data/info.pb
I20260808 08:27:34.471841 14055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:27:34.472903 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 13926
I20260808 08:27:34.473043 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786177493204855-23471-0/minicluster-data/master-2/wal/instance
I20260808 08:27:34.473369 23471 external_mini_cluster.cc:1428] Setting key 5798df20984c11d115ee3d90ac7ca8f2
I20260808 08:27:34.476080 14055 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38449 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 08:27:34 dist-test-slave-f7th krb5kdc[13657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.235.252@KRBTEST.COM
Aug 08 08:27:34 dist-test-slave-f7th krb5kdc[13657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177654, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.252@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:27:34.486675 14059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:34.474630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52494 (local address 127.22.235.252:34823)
0808 08:27:34.475010 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:34.475024 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:34.475058 (+ 34us) server_negotiation.cc:407] Connection header received
0808 08:27:34.475666 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:34.475681 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:34.475988 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:34.476257 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:27:34.477135 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:34.478186 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:34.479109 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:34.479772 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:34.479917 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:34.482422 (+ 2505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:27:34.482427 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:27:34.482436 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:27:34.482458 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:27:34.484035 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:34.484483 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:34.484489 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:34.484495 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:34.484646 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:34.484909 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:34.484911 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:34.484912 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:34.485284 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:27:34.485469 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:34.485707 (+ 238us) server_negotiation.cc:299] Negotiation successful
0808 08:27:34.485966 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":100,"threads_started":1}
I20260808 08:27:34.486936 13798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:34.477946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56720 (local address 127.22.235.254:38449)
0808 08:27:34.478070 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:34.478074 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:34.478318 (+ 244us) server_negotiation.cc:407] Connection header received
0808 08:27:34.478453 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:34.478456 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:34.478510 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:34.478584 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:27:34.479185 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:34.479899 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:34.481094 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:34.481316 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:34.481404 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:34.483678 (+ 2274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:27:34.483681 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:27:34.483682 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:27:34.483707 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:27:34.485191 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:34.485670 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:34.485674 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:34.485676 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:34.485764 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:34.486068 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:34.486071 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:34.486073 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:34.486260 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:27:34.486347 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:34.486665 (+ 318us) server_negotiation.cc:299] Negotiation successful
0808 08:27:34.486771 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 08:27:34.486977 14060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:34.477835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:38449 (local address 127.0.0.1:56720)
0808 08:27:34.478200 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:34.478247 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:34.478365 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:34.478644 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:27:34.478648 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:27:34.478947 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:27:34.479067 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:34.479073 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:34.480025 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:34.480029 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:34.480959 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:34.480982 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:34.481463 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:34.481467 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:34.481550 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:34.482043 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:27:34.482062 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:27:34.483540 (+ 1478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:27:34.485287 (+ 1747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:34.485294 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:34.485299 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:34.485572 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:34.485877 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:34.485880 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:34.485882 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:34.485977 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:34.486365 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:27:34.486371 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:27:34.486477 (+ 106us) client_negotiation.cc:769] Sending connection context
0808 08:27:34.486594 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 08:27:34.486766 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":142,"threads_started":1}
I20260808 08:27:34.488065 14055 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46733 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 08:27:34 dist-test-slave-f7th krb5kdc[13657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177654, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.252@KRBTEST.COM for kudu/127.22.235.253@KRBTEST.COM
Aug 08 08:27:34 dist-test-slave-f7th krb5kdc[13657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:27:34.495920 13925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:34.488629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50748 (local address 127.22.235.253:46733)
0808 08:27:34.488755 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:34.488759 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:34.488772 (+ 13us) server_negotiation.cc:407] Connection header received
0808 08:27:34.488811 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:34.488813 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:34.488849 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:34.488913 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:27:34.489318 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:34.489915 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:34.490775 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:34.491099 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:34.491203 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:34.492960 (+ 1757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:27:34.492964 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:27:34.492967 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:27:34.492996 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:27:34.494345 (+ 1349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:34.494772 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:34.494776 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:34.494777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:34.494869 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:34.495162 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:34.495165 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:34.495167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:34.495378 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:27:34.495451 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:34.495727 (+ 276us) server_negotiation.cc:299] Negotiation successful
0808 08:27:34.495812 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 08:27:34.495957 14060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:34.488503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.253:46733 (local address 127.0.0.1:50748)
0808 08:27:34.488611 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:34.488644 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:34.488719 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:34.488938 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:27:34.488940 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:27:34.489104 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:27:34.489213 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:34.489221 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:34.490025 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:34.490028 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:34.490654 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:34.490666 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:34.491212 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:34.491216 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:34.491291 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:34.491683 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:27:34.491694 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:27:34.492826 (+ 1132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:27:34.494433 (+ 1607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:34.494437 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:34.494438 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:34.494681 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:34.494959 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:34.494970 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:34.494972 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:34.495065 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:34.495466 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:27:34.495469 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:27:34.495566 (+ 97us) client_negotiation.cc:769] Sending connection context
0808 08:27:34.495678 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 08:27:34.495796 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 08:27:34.496989 14055 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.235.252" port: 34823 } has no permanent_uuid. Determining permanent_uuid...
I20260808 08:27:34.498172 13798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:34.489173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56722 (local address 127.22.235.254:38449)
0808 08:27:34.489272 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:34.489276 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:34.489393 (+ 117us) server_negotiation.cc:407] Connection header received
0808 08:27:34.489460 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:34.489463 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:34.489512 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:34.489586 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:27:34.490078 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:34.490608 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:34.491769 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:34.491984 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:34.492068 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:34.494435 (+ 2367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:27:34.494439 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:27:34.494441 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:27:34.494474 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:27:34.495964 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:34.496445 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:34.496447 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:34.496448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:34.496548 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:34.496843 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:34.496846 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:34.496848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:34.497057 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:27:34.497291 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:34.497922 (+ 631us) server_negotiation.cc:299] Negotiation successful
0808 08:27:34.498024 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
Aug 08 08:27:34 dist-test-slave-f7th krb5kdc[13657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177654, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.252@KRBTEST.COM for kudu/127.22.235.252@KRBTEST.COM
I20260808 08:27:34.502425 13915 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.235.252" port: 34823 } attempt: 3
Aug 08 08:27:34 dist-test-slave-f7th krb5kdc[13657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.235.253@KRBTEST.COM
I20260808 08:27:34.505318 14059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:34.497470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52510 (local address 127.22.235.252:34823)
0808 08:27:34.497577 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:34.497581 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:34.497593 (+ 12us) server_negotiation.cc:407] Connection header received
0808 08:27:34.497656 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:34.497658 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:34.497689 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:34.497743 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:27:34.498121 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:34.499122 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:34.499861 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:34.500189 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:34.500269 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:34.502060 (+ 1791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:27:34.502064 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:27:34.502066 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:27:34.502092 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:27:34.503690 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:34.504117 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:34.504121 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:34.504122 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:34.504206 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:34.504471 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:34.504474 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:34.504476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:34.504746 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:27:34.504837 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:34.505063 (+ 226us) server_negotiation.cc:299] Negotiation successful
0808 08:27:34.505170 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 08:27:34.505460 14060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:34.497413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:34823 (local address 127.0.0.1:52510)
0808 08:27:34.497505 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:34.497518 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:34.497583 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:34.497781 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:27:34.497785 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:27:34.497932 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:27:34.498030 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:34.498037 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:34.499230 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:34.499233 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:34.499750 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:34.499759 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:34.500283 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:34.500287 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:34.500364 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:34.500766 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:27:34.500779 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:27:34.501939 (+ 1160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:27:34.503779 (+ 1840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:34.503782 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:34.503784 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:34.504013 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:34.504308 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:34.504311 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:34.504313 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:34.504386 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:34.504826 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:27:34.504829 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:27:34.504911 (+ 82us) client_negotiation.cc:769] Sending connection context
0808 08:27:34.505004 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 08:27:34.505329 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
Aug 08 08:27:34 dist-test-slave-f7th krb5kdc[13657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177654, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.253@KRBTEST.COM for kudu/127.22.235.252@KRBTEST.COM
I20260808 08:27:34.507700 13925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:34.499270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50760 (local address 127.22.235.253:46733)
0808 08:27:34.499365 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:34.499369 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:34.499380 (+ 11us) server_negotiation.cc:407] Connection header received
0808 08:27:34.499473 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:34.499476 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:34.499523 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:34.499583 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:27:34.500193 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:34.500866 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:34.501557 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:34.501821 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:34.501925 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:34.504287 (+ 2362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:27:34.504291 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:27:34.504293 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:27:34.504316 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:27:34.506051 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:34.506504 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:34.506507 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:34.506509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:34.506584 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:34.506881 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:34.506885 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:34.506886 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:34.507082 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:27:34.507301 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:34.507443 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 08:27:34.507558 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 08:27:34.508852 14055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04f4161cd07846e4a4bf5ef1c6eb0b04: Bootstrap starting.
I20260808 08:27:34.509495 13791 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.235.252" port: 34823 } attempt: 3
I20260808 08:27:34.510923 23471 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 08:27:34.510982 13923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:34.502760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:34823 (local address 127.0.0.1:52518)
0808 08:27:34.502895 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:34.502909 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:34.502985 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:34.503406 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:27:34.503409 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:27:34.503562 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:27:34.503629 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:34.503634 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:34.504535 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:34.504539 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:34.505319 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:34.505330 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:34.505928 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:34.505932 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:34.506028 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:34.506406 (+ 378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:27:34.506419 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:27:34.507682 (+ 1263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:27:34.509445 (+ 1763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:34.509448 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:34.509451 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:34.509691 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:34.510011 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:34.510014 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:34.510016 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:34.510103 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:34.510562 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:27:34.510565 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:27:34.510654 (+ 89us) client_negotiation.cc:769] Sending connection context
0808 08:27:34.510734 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 08:27:34.510843 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 08:27:34.511579 14061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:34.502959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52518 (local address 127.22.235.252:34823)
0808 08:27:34.503224 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:34.503230 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:34.503241 (+ 11us) server_negotiation.cc:407] Connection header received
0808 08:27:34.503266 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:34.503268 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:34.503299 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:34.503372 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:27:34.503727 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:34.504414 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:34.505466 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:34.505803 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:34.506206 (+ 403us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:34.507820 (+ 1614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:27:34.507824 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:27:34.507826 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:27:34.507849 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:27:34.509351 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:34.509804 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:34.509807 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:34.509809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:34.509906 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:34.510212 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:34.510215 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:34.510217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:34.510438 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:27:34.511218 (+ 780us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:34.511287 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 08:27:34.511388 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":85,"threads_started":1}
I20260808 08:27:34.511708 14055 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04f4161cd07846e4a4bf5ef1c6eb0b04: Neither blocks nor log segments found. Creating new log.
I20260808 08:27:34.513280 14055 log.cc:824] T 00000000000000000000000000000000 P 04f4161cd07846e4a4bf5ef1c6eb0b04: Log is configured to *not* fsync() on all Append() calls
I20260808 08:27:34.514714 13915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 086b16b15ca94487b033ccdb71d73582: Bootstrap starting.
Aug 08 08:27:34 dist-test-slave-f7th krb5kdc[13657](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177653, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for kudu/127.22.235.252@KRBTEST.COM
I20260808 08:27:34.517388 13915 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 086b16b15ca94487b033ccdb71d73582: Neither blocks nor log segments found. Creating new log.
I20260808 08:27:34.518229 13915 log.cc:824] T 00000000000000000000000000000000 P 086b16b15ca94487b033ccdb71d73582: Log is configured to *not* fsync() on all Append() calls
I20260808 08:27:34.518615 13799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:34.509832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:34823 (local address 127.0.0.1:52520)
0808 08:27:34.510077 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:34.510090 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:34.510158 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:34.510567 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:27:34.510570 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:27:34.510740 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:27:34.510836 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:34.510843 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:34.512065 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:34.512069 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:34.512842 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:34.512855 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:34.513482 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:34.513486 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:34.513568 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:34.513972 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:27:34.513987 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:27:34.515306 (+ 1319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:27:34.517106 (+ 1800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:34.517109 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:34.517112 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:34.517303 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:34.517655 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:34.517658 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:34.517660 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:34.517747 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:34.518160 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:27:34.518163 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:27:34.518253 (+ 90us) client_negotiation.cc:769] Sending connection context
0808 08:27:34.518350 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 08:27:34.518469 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171}
I20260808 08:27:34.519248 14059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:34.510275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52520 (local address 127.22.235.252:34823)
0808 08:27:34.510369 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:34.510372 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:34.510382 (+ 10us) server_negotiation.cc:407] Connection header received
0808 08:27:34.510411 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:34.510413 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:34.510446 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:34.510936 (+ 490us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:27:34.511028 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:34.511817 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:34.513011 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:34.513351 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:34.513767 (+ 416us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:34.515454 (+ 1687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:27:34.515458 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:27:34.515460 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:27:34.515495 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:27:34.516981 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:34.517396 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:34.517399 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:34.517401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:34.517548 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:34.517843 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:34.517846 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:34.517848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:34.518041 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:27:34.518119 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:34.519015 (+ 896us) server_negotiation.cc:299] Negotiation successful
0808 08:27:34.519121 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 08:27:34.519757 14055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04f4161cd07846e4a4bf5ef1c6eb0b04: No bootstrap required, opened a new log
I20260808 08:27:34.522836 13791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e5c7889d202401daf1fb59c21742123: Bootstrap starting.
I20260808 08:27:34.523013 14055 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04f4161cd07846e4a4bf5ef1c6eb0b04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e5c7889d202401daf1fb59c21742123" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38449 } } peers { permanent_uuid: "086b16b15ca94487b033ccdb71d73582" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46733 } } peers { permanent_uuid: "04f4161cd07846e4a4bf5ef1c6eb0b04" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 34823 } }
I20260808 08:27:34.523208 14055 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04f4161cd07846e4a4bf5ef1c6eb0b04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:27:34.523260 14055 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04f4161cd07846e4a4bf5ef1c6eb0b04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04f4161cd07846e4a4bf5ef1c6eb0b04, State: Initialized, Role: FOLLOWER
I20260808 08:27:34.523676 14055 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04f4161cd07846e4a4bf5ef1c6eb0b04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e5c7889d202401daf1fb59c21742123" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38449 } } peers { permanent_uuid: "086b16b15ca94487b033ccdb71d73582" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46733 } } peers { permanent_uuid: "04f4161cd07846e4a4bf5ef1c6eb0b04" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 34823 } }
I20260808 08:27:34.524097 14082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04f4161cd07846e4a4bf5ef1c6eb0b04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e5c7889d202401daf1fb59c21742123" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38449 } } peers { permanent_uuid: "086b16b15ca94487b033ccdb71d73582" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46733 } } peers { permanent_uuid: "04f4161cd07846e4a4bf5ef1c6eb0b04" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 34823 } } }
I20260808 08:27:34.524296 14082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04f4161cd07846e4a4bf5ef1c6eb0b04 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 08:27:34.524372 13925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:34.514853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50762 (local address 127.22.235.253:46733)
0808 08:27:34.515084 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:34.515088 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:34.515568 (+ 480us) server_negotiation.cc:407] Connection header received
0808 08:27:34.515656 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:34.515659 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:34.515698 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:34.515768 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:27:34.516668 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:34.517340 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:34.518442 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:34.518737 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:34.518811 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:34.520423 (+ 1612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:27:34.520426 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:27:34.520429 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:27:34.520452 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:27:34.521916 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:34.522388 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:34.522392 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:34.522394 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:34.522474 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:34.522763 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:34.522766 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:34.522767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:34.522957 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:27:34.523082 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:34.524132 (+ 1050us) server_negotiation.cc:299] Negotiation successful
0808 08:27:34.524241 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163}
I20260808 08:27:34.525125 14055 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04f4161cd07846e4a4bf5ef1c6eb0b04 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:27:34.525473 13791 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e5c7889d202401daf1fb59c21742123: Neither blocks nor log segments found. Creating new log.
I20260808 08:27:34.526311 13791 log.cc:824] T 00000000000000000000000000000000 P 8e5c7889d202401daf1fb59c21742123: Log is configured to *not* fsync() on all Append() calls
I20260808 08:27:34.530867 13915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 086b16b15ca94487b033ccdb71d73582: No bootstrap required, opened a new log
I20260808 08:27:34.531735 13798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:34.515050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56736 (local address 127.22.235.254:38449)
0808 08:27:34.515796 (+ 746us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:34.515801 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:34.515813 (+ 12us) server_negotiation.cc:407] Connection header received
0808 08:27:34.515872 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:34.515875 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:34.516181 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:34.516280 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:27:34.517444 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:34.518113 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:34.525597 (+ 7484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:34.525851 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:34.525941 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:34.527541 (+ 1600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:27:34.527545 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:27:34.527548 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:27:34.527577 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:27:34.529372 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:34.530066 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:34.530069 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:34.530071 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:34.530157 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:34.530690 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:34.530694 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:34.530695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:34.530896 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:27:34.531088 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:34.531474 (+ 386us) server_negotiation.cc:299] Negotiation successful
0808 08:27:34.531584 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":667}
I20260808 08:27:34.532963 14061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:34.514479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52530 (local address 127.22.235.252:34823)
0808 08:27:34.514854 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:34.514858 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:34.523835 (+ 8977us) server_negotiation.cc:407] Connection header received
0808 08:27:34.523874 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:34.523876 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:34.523916 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:34.523982 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:27:34.526094 (+ 2112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:34.526776 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:34.527772 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:34.528115 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:34.528213 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:34.529603 (+ 1390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:27:34.529607 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:27:34.529609 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:27:34.529632 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:27:34.531088 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:34.531558 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:34.531561 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:34.531563 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:34.531645 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:34.531932 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:34.531935 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:34.531937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:34.532139 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:27:34.532273 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:34.532714 (+ 441us) server_negotiation.cc:299] Negotiation successful
0808 08:27:34.532826 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307}
I20260808 08:27:34.530328 13926 master_runner.cc:186] Error getting master registration for 127.22.235.254:38449: OK, instance_id { permanent_uuid: "8e5c7889d202401daf1fb59c21742123" instance_seqno: 1786177654048561 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 08:27:34.533246 13926 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 08:27:34.535440 13915 raft_consensus.cc:348] T 00000000000000000000000000000000 P 086b16b15ca94487b033ccdb71d73582 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e5c7889d202401daf1fb59c21742123" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38449 } } peers { permanent_uuid: "086b16b15ca94487b033ccdb71d73582" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46733 } } peers { permanent_uuid: "04f4161cd07846e4a4bf5ef1c6eb0b04" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 34823 } }
I20260808 08:27:34.535661 13915 raft_consensus.cc:374] T 00000000000000000000000000000000 P 086b16b15ca94487b033ccdb71d73582 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:27:34.535724 13915 raft_consensus.cc:729] T 00000000000000000000000000000000 P 086b16b15ca94487b033ccdb71d73582 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 086b16b15ca94487b033ccdb71d73582, State: Initialized, Role: FOLLOWER
I20260808 08:27:34.536203 13915 consensus_queue.cc:260] T 00000000000000000000000000000000 P 086b16b15ca94487b033ccdb71d73582 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e5c7889d202401daf1fb59c21742123" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38449 } } peers { permanent_uuid: "086b16b15ca94487b033ccdb71d73582" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46733 } } peers { permanent_uuid: "04f4161cd07846e4a4bf5ef1c6eb0b04" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 34823 } }
I20260808 08:27:34.536711 14108 sys_catalog.cc:455] T 00000000000000000000000000000000 P 086b16b15ca94487b033ccdb71d73582 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e5c7889d202401daf1fb59c21742123" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38449 } } peers { permanent_uuid: "086b16b15ca94487b033ccdb71d73582" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46733 } } peers { permanent_uuid: "04f4161cd07846e4a4bf5ef1c6eb0b04" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 34823 } } }
W20260808 08:27:34.536825 14104 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 04f4161cd07846e4a4bf5ef1c6eb0b04: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 08:27:34.536892 14104 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 08:27:34.536978 14108 sys_catalog.cc:458] T 00000000000000000000000000000000 P 086b16b15ca94487b033ccdb71d73582 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 08:27:34.537447 13915 sys_catalog.cc:564] T 00000000000000000000000000000000 P 086b16b15ca94487b033ccdb71d73582 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:27:34.541064 13791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e5c7889d202401daf1fb59c21742123: No bootstrap required, opened a new log
W20260808 08:27:34.543025 14129 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 086b16b15ca94487b033ccdb71d73582: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 08:27:34.543104 14129 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 08:27:34.544432 13791 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e5c7889d202401daf1fb59c21742123 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e5c7889d202401daf1fb59c21742123" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38449 } } peers { permanent_uuid: "086b16b15ca94487b033ccdb71d73582" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46733 } } peers { permanent_uuid: "04f4161cd07846e4a4bf5ef1c6eb0b04" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 34823 } }
I20260808 08:27:34.544597 13791 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e5c7889d202401daf1fb59c21742123 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:27:34.545437 13791 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e5c7889d202401daf1fb59c21742123 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e5c7889d202401daf1fb59c21742123, State: Initialized, Role: FOLLOWER
I20260808 08:27:34.545919 13791 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e5c7889d202401daf1fb59c21742123 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e5c7889d202401daf1fb59c21742123" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38449 } } peers { permanent_uuid: "086b16b15ca94487b033ccdb71d73582" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46733 } } peers { permanent_uuid: "04f4161cd07846e4a4bf5ef1c6eb0b04" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 34823 } }
I20260808 08:27:34.547245 13791 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e5c7889d202401daf1fb59c21742123 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:27:34.545368 13802 master_runner.cc:186] Error getting master registration for 127.22.235.254:38449: OK, instance_id { permanent_uuid: "8e5c7889d202401daf1fb59c21742123" instance_seqno: 1786177654048561 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 08:27:34.548445 13802 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 08:27:34.548718 14134 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e5c7889d202401daf1fb59c21742123 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e5c7889d202401daf1fb59c21742123" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38449 } } peers { permanent_uuid: "086b16b15ca94487b033ccdb71d73582" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46733 } } peers { permanent_uuid: "04f4161cd07846e4a4bf5ef1c6eb0b04" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 34823 } } }
I20260808 08:27:34.548848 14134 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e5c7889d202401daf1fb59c21742123 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 08:27:34.553614 14151 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8e5c7889d202401daf1fb59c21742123: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 08:27:34.553695 14151 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 08:27:34.558099 13682 master_runner.cc:263] No leader master found from master 8e5c7889d202401daf1fb59c21742123
I20260808 08:27:34.558171 13682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 08:27:34.678671 14082 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04f4161cd07846e4a4bf5ef1c6eb0b04 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 08:27:34.678877 14082 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04f4161cd07846e4a4bf5ef1c6eb0b04 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e5c7889d202401daf1fb59c21742123" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38449 } } peers { permanent_uuid: "086b16b15ca94487b033ccdb71d73582" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46733 } } peers { permanent_uuid: "04f4161cd07846e4a4bf5ef1c6eb0b04" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 34823 } }
I20260808 08:27:34.679853 14082 leader_election.cc:288] T 00000000000000000000000000000000 P 04f4161cd07846e4a4bf5ef1c6eb0b04 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e5c7889d202401daf1fb59c21742123 (127.22.235.254:38449), 086b16b15ca94487b033ccdb71d73582 (127.22.235.253:46733)
I20260808 08:27:34.681147 13872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "04f4161cd07846e4a4bf5ef1c6eb0b04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "086b16b15ca94487b033ccdb71d73582" is_pre_election: true
I20260808 08:27:34.681454 13872 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 086b16b15ca94487b033ccdb71d73582 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04f4161cd07846e4a4bf5ef1c6eb0b04 in term 0.
I20260808 08:27:34.681859 13966 leader_election.cc:302] T 00000000000000000000000000000000 P 04f4161cd07846e4a4bf5ef1c6eb0b04 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04f4161cd07846e4a4bf5ef1c6eb0b04, 086b16b15ca94487b033ccdb71d73582; no voters:
I20260808 08:27:34.680521 13745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "04f4161cd07846e4a4bf5ef1c6eb0b04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e5c7889d202401daf1fb59c21742123" is_pre_election: true
I20260808 08:27:34.682353 13745 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8e5c7889d202401daf1fb59c21742123 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04f4161cd07846e4a4bf5ef1c6eb0b04 in term 0.
I20260808 08:27:34.682921 14082 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04f4161cd07846e4a4bf5ef1c6eb0b04 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 08:27:34.683009 14082 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04f4161cd07846e4a4bf5ef1c6eb0b04 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 08:27:34.683054 14082 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04f4161cd07846e4a4bf5ef1c6eb0b04 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:27:34.684399 14082 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04f4161cd07846e4a4bf5ef1c6eb0b04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e5c7889d202401daf1fb59c21742123" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38449 } } peers { permanent_uuid: "086b16b15ca94487b033ccdb71d73582" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46733 } } peers { permanent_uuid: "04f4161cd07846e4a4bf5ef1c6eb0b04" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 34823 } }
I20260808 08:27:34.684931 14082 leader_election.cc:288] T 00000000000000000000000000000000 P 04f4161cd07846e4a4bf5ef1c6eb0b04 [CANDIDATE]: Term 1 election: Requested vote from peers 8e5c7889d202401daf1fb59c21742123 (127.22.235.254:38449), 086b16b15ca94487b033ccdb71d73582 (127.22.235.253:46733)
I20260808 08:27:34.685580 13872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "04f4161cd07846e4a4bf5ef1c6eb0b04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "086b16b15ca94487b033ccdb71d73582"
I20260808 08:27:34.685706 13872 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 086b16b15ca94487b033ccdb71d73582 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:27:34.687022 13872 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 086b16b15ca94487b033ccdb71d73582 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04f4161cd07846e4a4bf5ef1c6eb0b04 in term 1.
I20260808 08:27:34.687391 13745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "04f4161cd07846e4a4bf5ef1c6eb0b04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e5c7889d202401daf1fb59c21742123"
I20260808 08:27:34.687520 13745 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e5c7889d202401daf1fb59c21742123 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:27:34.688818 13745 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8e5c7889d202401daf1fb59c21742123 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04f4161cd07846e4a4bf5ef1c6eb0b04 in term 1.
I20260808 08:27:34.689217 13961 leader_election.cc:302] T 00000000000000000000000000000000 P 04f4161cd07846e4a4bf5ef1c6eb0b04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04f4161cd07846e4a4bf5ef1c6eb0b04, 8e5c7889d202401daf1fb59c21742123; no voters:
I20260808 08:27:34.689585 14082 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04f4161cd07846e4a4bf5ef1c6eb0b04 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:27:34.689815 14082 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04f4161cd07846e4a4bf5ef1c6eb0b04 [term 1 LEADER]: Becoming Leader. State: Replica: 04f4161cd07846e4a4bf5ef1c6eb0b04, State: Running, Role: LEADER
I20260808 08:27:34.690171 14082 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04f4161cd07846e4a4bf5ef1c6eb0b04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e5c7889d202401daf1fb59c21742123" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38449 } } peers { permanent_uuid: "086b16b15ca94487b033ccdb71d73582" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46733 } } peers { permanent_uuid: "04f4161cd07846e4a4bf5ef1c6eb0b04" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 34823 } }
I20260808 08:27:34.692509 14082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04f4161cd07846e4a4bf5ef1c6eb0b04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04f4161cd07846e4a4bf5ef1c6eb0b04. Latest consensus state: current_term: 1 leader_uuid: "04f4161cd07846e4a4bf5ef1c6eb0b04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e5c7889d202401daf1fb59c21742123" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38449 } } peers { permanent_uuid: "086b16b15ca94487b033ccdb71d73582" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46733 } } peers { permanent_uuid: "04f4161cd07846e4a4bf5ef1c6eb0b04" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 34823 } } }
I20260808 08:27:34.692651 14082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04f4161cd07846e4a4bf5ef1c6eb0b04 [sys.catalog]: This master's current role is: LEADER
I20260808 08:27:34.693037 14155 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:27:34.693969 14155 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:27:34.700646 13872 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 086b16b15ca94487b033ccdb71d73582 [term 1 FOLLOWER]: Refusing update from remote peer 04f4161cd07846e4a4bf5ef1c6eb0b04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:27:34.701288 14153 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 04f4161cd07846e4a4bf5ef1c6eb0b04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "086b16b15ca94487b033ccdb71d73582" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:27:34.705312 14108 sys_catalog.cc:455] T 00000000000000000000000000000000 P 086b16b15ca94487b033ccdb71d73582 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04f4161cd07846e4a4bf5ef1c6eb0b04. Latest consensus state: current_term: 1 leader_uuid: "04f4161cd07846e4a4bf5ef1c6eb0b04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e5c7889d202401daf1fb59c21742123" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38449 } } peers { permanent_uuid: "086b16b15ca94487b033ccdb71d73582" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46733 } } peers { permanent_uuid: "04f4161cd07846e4a4bf5ef1c6eb0b04" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 34823 } } }
I20260808 08:27:34.705433 14108 sys_catalog.cc:458] T 00000000000000000000000000000000 P 086b16b15ca94487b033ccdb71d73582 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 08:27:34.707824 13745 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8e5c7889d202401daf1fb59c21742123 [term 1 FOLLOWER]: Refusing update from remote peer 04f4161cd07846e4a4bf5ef1c6eb0b04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:27:34.708374 14153 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 04f4161cd07846e4a4bf5ef1c6eb0b04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e5c7889d202401daf1fb59c21742123" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:27:34.712090 14134 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e5c7889d202401daf1fb59c21742123 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04f4161cd07846e4a4bf5ef1c6eb0b04. Latest consensus state: current_term: 1 leader_uuid: "04f4161cd07846e4a4bf5ef1c6eb0b04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e5c7889d202401daf1fb59c21742123" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38449 } } peers { permanent_uuid: "086b16b15ca94487b033ccdb71d73582" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46733 } } peers { permanent_uuid: "04f4161cd07846e4a4bf5ef1c6eb0b04" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 34823 } } }
I20260808 08:27:34.712214 14134 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e5c7889d202401daf1fb59c21742123 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 08:27:34.712874 13970 tablet.cc:2369] T 00000000000000000000000000000000 P 04f4161cd07846e4a4bf5ef1c6eb0b04: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 08:27:34.714211 14153 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04f4161cd07846e4a4bf5ef1c6eb0b04 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "04f4161cd07846e4a4bf5ef1c6eb0b04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e5c7889d202401daf1fb59c21742123" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38449 } } peers { permanent_uuid: "086b16b15ca94487b033ccdb71d73582" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46733 } } peers { permanent_uuid: "04f4161cd07846e4a4bf5ef1c6eb0b04" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 34823 } } }
I20260808 08:27:34.714323 14153 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04f4161cd07846e4a4bf5ef1c6eb0b04 [sys.catalog]: This master's current role is: LEADER
I20260808 08:27:34.715250 14159 mvcc.cc:205] Tried to move back new op lower bound from 7316183673646333952 to 7316183673614393344. Current Snapshot: MvccSnapshot[applied={T|T < 7316183673646333952}]
I20260808 08:27:34.715404 14108 sys_catalog.cc:455] T 00000000000000000000000000000000 P 086b16b15ca94487b033ccdb71d73582 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "04f4161cd07846e4a4bf5ef1c6eb0b04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e5c7889d202401daf1fb59c21742123" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38449 } } peers { permanent_uuid: "086b16b15ca94487b033ccdb71d73582" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46733 } } peers { permanent_uuid: "04f4161cd07846e4a4bf5ef1c6eb0b04" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 34823 } } }
I20260808 08:27:34.715507 14108 sys_catalog.cc:458] T 00000000000000000000000000000000 P 086b16b15ca94487b033ccdb71d73582 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 08:27:34.715925 14156 mvcc.cc:205] Tried to move back new op lower bound from 7316183673646333952 to 7316183673614393344. Current Snapshot: MvccSnapshot[applied={T|T < 7316183673646333952 or (T in {7316183673646333952})}]
I20260808 08:27:34.716214 14155 catalog_manager.cc:1347] Generated new cluster ID: 5cb1f19aaf6045c7a540527576e309c9
I20260808 08:27:34.716271 14155 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:27:34.722601 14082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04f4161cd07846e4a4bf5ef1c6eb0b04 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "04f4161cd07846e4a4bf5ef1c6eb0b04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e5c7889d202401daf1fb59c21742123" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38449 } } peers { permanent_uuid: "086b16b15ca94487b033ccdb71d73582" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46733 } } peers { permanent_uuid: "04f4161cd07846e4a4bf5ef1c6eb0b04" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 34823 } } }
I20260808 08:27:34.722721 14082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04f4161cd07846e4a4bf5ef1c6eb0b04 [sys.catalog]: This master's current role is: LEADER
I20260808 08:27:34.724011 14134 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e5c7889d202401daf1fb59c21742123 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "04f4161cd07846e4a4bf5ef1c6eb0b04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e5c7889d202401daf1fb59c21742123" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38449 } } peers { permanent_uuid: "086b16b15ca94487b033ccdb71d73582" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46733 } } peers { permanent_uuid: "04f4161cd07846e4a4bf5ef1c6eb0b04" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 34823 } } }
I20260808 08:27:34.724134 14134 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e5c7889d202401daf1fb59c21742123 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 08:27:34.724426 14161 mvcc.cc:205] Tried to move back new op lower bound from 7316183673646333952 to 7316183673614393344. Current Snapshot: MvccSnapshot[applied={T|T < 7316183673646333952}]
I20260808 08:27:34.733480 14155 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:27:34.734750 14155 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:27:34.744159 14155 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 04f4161cd07846e4a4bf5ef1c6eb0b04: Generated new TSK 0
I20260808 08:27:34.744899 14155 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 08:27:35.544390 14129 catalog_manager.cc:1259] Loaded cluster ID: 5cb1f19aaf6045c7a540527576e309c9
I20260808 08:27:35.544471 14129 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 086b16b15ca94487b033ccdb71d73582: loading cluster ID for follower catalog manager: success
I20260808 08:27:35.546149 14129 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 086b16b15ca94487b033ccdb71d73582: acquiring CA information for follower catalog manager: success
I20260808 08:27:35.547535 14129 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 086b16b15ca94487b033ccdb71d73582: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 08:27:35.554795 14151 catalog_manager.cc:1259] Loaded cluster ID: 5cb1f19aaf6045c7a540527576e309c9
I20260808 08:27:35.554854 14151 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8e5c7889d202401daf1fb59c21742123: loading cluster ID for follower catalog manager: success
I20260808 08:27:35.556363 14151 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8e5c7889d202401daf1fb59c21742123: acquiring CA information for follower catalog manager: success
I20260808 08:27:35.557426 14151 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8e5c7889d202401daf1fb59c21742123: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 08:27:35.813913 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 13682
I20260808 08:27:35.828995 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 13802
I20260808 08:27:35.839030 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 13926
2026-08-08T08:27:35Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3283 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786177493204855-23471-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 08:27:35 dist-test-slave-f7th krb5kdc[14177](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 08:27:35 dist-test-slave-f7th krb5kdc[14177](info): set up 2 sockets
Aug 08 08:27:35 dist-test-slave-f7th krb5kdc[14177](info): commencing operation
krb5kdc: starting...
W20260808 08:27:37.016357 23471 mini_kdc.cc:121] Time spent starting KDC: real 1.144s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Aug 08 08:27:37 dist-test-slave-f7th krb5kdc[14177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T08:27:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:27:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.235.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
I20260808 08:27:37.135210 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:41901
--webserver_interface=127.22.235.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:41757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:41901
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:27:37.245756 14202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:27:37.246074 14202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:27:37.246135 14202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:27:37.251886 14202 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:27:37.251959 14202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:27:37.252000 14202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:27:37.252069 14202 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:27:37.252105 14202 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:27:37.257793 14202 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:41757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:41901
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:41901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.235.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14202
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:27:37.258765 14202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:27:37.259745 14202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:27:37.268115 14211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:27:37.268154 14210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:27:37.268154 14213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:27:37.268441 14202 server_base.cc:1034] running on GCE node
I20260808 08:27:37.269006 14202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:27:37.270056 14202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:27:37.271214 14202 hybrid_clock.cc:648] HybridClock initialized: now 1786177657271194 us; error 31 us; skew 500 ppm
Aug 08 08:27:37 dist-test-slave-f7th krb5kdc[14177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177657, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:27:37.273778 14202 init.cc:377] Logged in from keytab as kudu/127.22.235.254@KRBTEST.COM (short username kudu)
I20260808 08:27:37.274574 14202 webserver.cc:460] Webserver started at http://127.22.235.254:42009/ using document root <none> and password file <none>
I20260808 08:27:37.275080 14202 fs_manager.cc:359] Metadata directory not provided
I20260808 08:27:37.275136 14202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:27:37.275302 14202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:27:37.277271 14202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "606e4cff5e054017a326adc292fc1788"
format_stamp: "Formatted at 2026-08-08 08:27:37 on dist-test-slave-f7th"
server_key: "b9a966f1bb8dc93deb295b28ae0d7052"
server_key_iv: "c25a5a018c725ca5c5a7a87e8714e6a1"
server_key_version: "encryptionkey@0"
I20260808 08:27:37.277755 14202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "606e4cff5e054017a326adc292fc1788"
format_stamp: "Formatted at 2026-08-08 08:27:37 on dist-test-slave-f7th"
server_key: "b9a966f1bb8dc93deb295b28ae0d7052"
server_key_iv: "c25a5a018c725ca5c5a7a87e8714e6a1"
server_key_version: "encryptionkey@0"
I20260808 08:27:37.281381 14202 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 08:27:37.283777 14225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:37.285053 14202 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:27:37.285178 14202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "606e4cff5e054017a326adc292fc1788"
format_stamp: "Formatted at 2026-08-08 08:27:37 on dist-test-slave-f7th"
server_key: "b9a966f1bb8dc93deb295b28ae0d7052"
server_key_iv: "c25a5a018c725ca5c5a7a87e8714e6a1"
server_key_version: "encryptionkey@0"
I20260808 08:27:37.285279 14202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:27:37.299315 14202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:27:37.302460 14202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:27:37.302654 14202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:27:37.309319 14202 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.254:41901
I20260808 08:27:37.309357 14325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.254:41901 every 8 connection(s)
I20260808 08:27:37.310281 14202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:27:37.310940 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 14202
I20260808 08:27:37.311059 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:27:37.311354 23471 external_mini_cluster.cc:1428] Setting key 93834cdb91a7e317c103710284275a78
I20260808 08:27:37.313593 14326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:27:37.319020 14326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 606e4cff5e054017a326adc292fc1788: Bootstrap starting.
Aug 08 08:27:37 dist-test-slave-f7th krb5kdc[14177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:27:37.321416 14326 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 606e4cff5e054017a326adc292fc1788: Neither blocks nor log segments found. Creating new log.
I20260808 08:27:37.322109 14326 log.cc:824] T 00000000000000000000000000000000 P 606e4cff5e054017a326adc292fc1788: Log is configured to *not* fsync() on all Append() calls
I20260808 08:27:37.323915 14326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 606e4cff5e054017a326adc292fc1788: No bootstrap required, opened a new log
I20260808 08:27:37.324719 14329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:37.312760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50658 (local address 127.22.235.254:41901)
0808 08:27:37.313196 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:37.313210 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:37.313247 (+ 37us) server_negotiation.cc:407] Connection header received
0808 08:27:37.313914 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:37.313933 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:37.314267 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:37.314574 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:27:37.315377 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:37.316389 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:37.317271 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:37.317726 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:37.317875 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:37.320376 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:27:37.320382 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:27:37.320394 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:27:37.320422 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:27:37.321998 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:37.322443 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:37.322448 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:37.322453 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:37.322576 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:37.322835 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:37.322839 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:37.322840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:37.323210 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:27:37.323396 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:37.323670 (+ 274us) server_negotiation.cc:299] Negotiation successful
0808 08:27:37.323945 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":140,"threads_started":1}
I20260808 08:27:37.327543 23471 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 08:27:37.327814 14326 raft_consensus.cc:348] T 00000000000000000000000000000000 P 606e4cff5e054017a326adc292fc1788 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "606e4cff5e054017a326adc292fc1788" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 41901 } }
I20260808 08:27:37.328037 14326 raft_consensus.cc:374] T 00000000000000000000000000000000 P 606e4cff5e054017a326adc292fc1788 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:27:37.328104 14326 raft_consensus.cc:729] T 00000000000000000000000000000000 P 606e4cff5e054017a326adc292fc1788 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 606e4cff5e054017a326adc292fc1788, State: Initialized, Role: FOLLOWER
I20260808 08:27:37.328534 14326 consensus_queue.cc:260] T 00000000000000000000000000000000 P 606e4cff5e054017a326adc292fc1788 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "606e4cff5e054017a326adc292fc1788" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 41901 } }
I20260808 08:27:37.328716 14326 raft_consensus.cc:388] T 00000000000000000000000000000000 P 606e4cff5e054017a326adc292fc1788 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:27:37.328785 14326 raft_consensus.cc:482] T 00000000000000000000000000000000 P 606e4cff5e054017a326adc292fc1788 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:27:37.328886 14326 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 606e4cff5e054017a326adc292fc1788 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:27:37.329735 14326 raft_consensus.cc:504] T 00000000000000000000000000000000 P 606e4cff5e054017a326adc292fc1788 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "606e4cff5e054017a326adc292fc1788" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 41901 } }
I20260808 08:27:37.330060 14326 leader_election.cc:302] T 00000000000000000000000000000000 P 606e4cff5e054017a326adc292fc1788 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 606e4cff5e054017a326adc292fc1788; no voters:
I20260808 08:27:37.330349 14326 leader_election.cc:288] T 00000000000000000000000000000000 P 606e4cff5e054017a326adc292fc1788 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:27:37.330449 14331 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 606e4cff5e054017a326adc292fc1788 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:27:37.330667 14331 raft_consensus.cc:686] T 00000000000000000000000000000000 P 606e4cff5e054017a326adc292fc1788 [term 1 LEADER]: Becoming Leader. State: Replica: 606e4cff5e054017a326adc292fc1788, State: Running, Role: LEADER
I20260808 08:27:37.331004 14331 consensus_queue.cc:237] T 00000000000000000000000000000000 P 606e4cff5e054017a326adc292fc1788 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "606e4cff5e054017a326adc292fc1788" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 41901 } }
I20260808 08:27:37.331475 14326 sys_catalog.cc:564] T 00000000000000000000000000000000 P 606e4cff5e054017a326adc292fc1788 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:27:37.332602 14334 sys_catalog.cc:455] T 00000000000000000000000000000000 P 606e4cff5e054017a326adc292fc1788 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 606e4cff5e054017a326adc292fc1788. Latest consensus state: current_term: 1 leader_uuid: "606e4cff5e054017a326adc292fc1788" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "606e4cff5e054017a326adc292fc1788" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 41901 } } }
I20260808 08:27:37.332777 14334 sys_catalog.cc:458] T 00000000000000000000000000000000 P 606e4cff5e054017a326adc292fc1788 [sys.catalog]: This master's current role is: LEADER
I20260808 08:27:37.332641 14333 sys_catalog.cc:455] T 00000000000000000000000000000000 P 606e4cff5e054017a326adc292fc1788 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "606e4cff5e054017a326adc292fc1788" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "606e4cff5e054017a326adc292fc1788" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 41901 } } }
I20260808 08:27:37.332948 14333 sys_catalog.cc:458] T 00000000000000000000000000000000 P 606e4cff5e054017a326adc292fc1788 [sys.catalog]: This master's current role is: LEADER
I20260808 08:27:37.333841 14342 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:27:37.336416 14342 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:27:37.341993 14342 catalog_manager.cc:1347] Generated new cluster ID: ecf9ecefa0284c61b23768d4bfa02911
I20260808 08:27:37.342048 14342 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:27:37.358812 14342 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:27:37.359669 14342 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:27:37.364375 14342 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 606e4cff5e054017a326adc292fc1788: Generated new TSK 0
I20260808 08:27:37.364997 14342 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 08:27:37.383610 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 14202
2026-08-08T08:27:37Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1539 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 08:27:37 dist-test-slave-f7th krb5kdc[14368](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 08:27:37 dist-test-slave-f7th krb5kdc[14368](info): set up 2 sockets
Aug 08 08:27:37 dist-test-slave-f7th krb5kdc[14368](info): commencing operation
krb5kdc: starting...
W20260808 08:27:38.558446 23471 mini_kdc.cc:121] Time spent starting KDC: real 1.147s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Aug 08 08:27:38 dist-test-slave-f7th krb5kdc[14368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177658, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T08:27:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:27:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.235.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
I20260808 08:27:38.677162 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:46193
--webserver_interface=127.22.235.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:40861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:46193
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:27:38.790539 14391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:27:38.790829 14391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:27:38.790890 14391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:27:38.794926 14391 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:27:38.794994 14391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:27:38.795023 14391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:27:38.795045 14391 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:27:38.795070 14391 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:27:38.800402 14391 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:40861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:46193
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:46193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.235.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14391
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:27:38.801510 14391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:27:38.802538 14391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:27:38.810827 14398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:27:38.810876 14391 server_base.cc:1034] running on GCE node
W20260808 08:27:38.810802 14406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:27:38.810811 14400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:27:38.811612 14391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:27:38.812737 14391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:27:38.813897 14391 hybrid_clock.cc:648] HybridClock initialized: now 1786177658813882 us; error 27 us; skew 500 ppm
Aug 08 08:27:38 dist-test-slave-f7th krb5kdc[14368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177658, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:27:38.819445 14391 init.cc:377] Logged in from keytab as kudu/127.22.235.254@KRBTEST.COM (short username kudu)
I20260808 08:27:38.820307 14391 webserver.cc:460] Webserver started at http://127.22.235.254:33717/ using document root <none> and password file <none>
I20260808 08:27:38.820871 14391 fs_manager.cc:359] Metadata directory not provided
I20260808 08:27:38.820933 14391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:27:38.821120 14391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:27:38.823132 14391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "44f372ca2df24ab08e3f6ffa192eb3e1"
format_stamp: "Formatted at 2026-08-08 08:27:38 on dist-test-slave-f7th"
server_key: "10dbbed96ad8730cc7fd8ed49f6c1c7d"
server_key_iv: "1682069a614bb82e2ee2706e29085580"
server_key_version: "encryptionkey@0"
I20260808 08:27:38.823626 14391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "44f372ca2df24ab08e3f6ffa192eb3e1"
format_stamp: "Formatted at 2026-08-08 08:27:38 on dist-test-slave-f7th"
server_key: "10dbbed96ad8730cc7fd8ed49f6c1c7d"
server_key_iv: "1682069a614bb82e2ee2706e29085580"
server_key_version: "encryptionkey@0"
I20260808 08:27:38.827248 14391 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 08:27:38.829619 14421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:38.830783 14391 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:27:38.830893 14391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "44f372ca2df24ab08e3f6ffa192eb3e1"
format_stamp: "Formatted at 2026-08-08 08:27:38 on dist-test-slave-f7th"
server_key: "10dbbed96ad8730cc7fd8ed49f6c1c7d"
server_key_iv: "1682069a614bb82e2ee2706e29085580"
server_key_version: "encryptionkey@0"
I20260808 08:27:38.831002 14391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:27:38.847932 14391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:27:38.851137 14391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:27:38.851405 14391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:27:38.865389 14391 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.254:46193
I20260808 08:27:38.865712 14505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.254:46193 every 8 connection(s)
I20260808 08:27:38.866837 14391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:27:38.870239 14507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:27:38.873035 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 14391
I20260808 08:27:38.873149 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:27:38.873442 23471 external_mini_cluster.cc:1428] Setting key 3af194f340f25926edd7a4feb5463657
I20260808 08:27:38.878373 14507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1: Bootstrap starting.
I20260808 08:27:38.881233 14507 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1: Neither blocks nor log segments found. Creating new log.
I20260808 08:27:38.881954 14507 log.cc:824] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1: Log is configured to *not* fsync() on all Append() calls
Aug 08 08:27:38 dist-test-slave-f7th krb5kdc[14368](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177658, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:27:38.886082 14507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1: No bootstrap required, opened a new log
I20260808 08:27:38.890686 14507 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44f372ca2df24ab08e3f6ffa192eb3e1" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46193 } }
I20260808 08:27:38.890981 14507 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:27:38.891019 14510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:38.875087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38670 (local address 127.22.235.254:46193)
0808 08:27:38.875538 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:38.875558 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:38.875598 (+ 40us) server_negotiation.cc:407] Connection header received
0808 08:27:38.876440 (+ 842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:38.876468 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:38.876960 (+ 492us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:38.878435 (+ 1475us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:27:38.878901 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:38.880418 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:38.881883 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:38.882758 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:38.882929 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:38.885901 (+ 2972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:27:38.885908 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:27:38.885924 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:27:38.885955 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:27:38.887945 (+ 1990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:38.888591 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:38.888597 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:38.888604 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:38.888751 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:38.889039 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:38.889042 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:38.889044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:38.889462 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:27:38.889659 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:38.889946 (+ 287us) server_negotiation.cc:299] Negotiation successful
0808 08:27:38.890273 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":135,"threads_started":1}
I20260808 08:27:38.891042 14507 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44f372ca2df24ab08e3f6ffa192eb3e1, State: Initialized, Role: FOLLOWER
I20260808 08:27:38.891654 14507 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44f372ca2df24ab08e3f6ffa192eb3e1" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46193 } }
I20260808 08:27:38.891825 14507 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:27:38.891897 14507 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:27:38.892004 14507 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:27:38.893157 14507 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44f372ca2df24ab08e3f6ffa192eb3e1" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46193 } }
I20260808 08:27:38.893546 14507 leader_election.cc:302] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44f372ca2df24ab08e3f6ffa192eb3e1; no voters:
I20260808 08:27:38.893882 14507 leader_election.cc:288] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:27:38.893971 14512 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:27:38.894179 14512 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [term 1 LEADER]: Becoming Leader. State: Replica: 44f372ca2df24ab08e3f6ffa192eb3e1, State: Running, Role: LEADER
I20260808 08:27:38.894502 14512 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44f372ca2df24ab08e3f6ffa192eb3e1" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46193 } }
I20260808 08:27:38.895226 14507 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:27:38.895711 14512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44f372ca2df24ab08e3f6ffa192eb3e1. Latest consensus state: current_term: 1 leader_uuid: "44f372ca2df24ab08e3f6ffa192eb3e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44f372ca2df24ab08e3f6ffa192eb3e1" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46193 } } }
I20260808 08:27:38.895823 14512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [sys.catalog]: This master's current role is: LEADER
I20260808 08:27:38.896143 14513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44f372ca2df24ab08e3f6ffa192eb3e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44f372ca2df24ab08e3f6ffa192eb3e1" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46193 } } }
I20260808 08:27:38.896229 14513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [sys.catalog]: This master's current role is: LEADER
I20260808 08:27:38.896937 14519 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:27:38.900089 14519 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:27:38.906124 14519 catalog_manager.cc:1347] Generated new cluster ID: bd2ab53661ea4a199c69e609adbf39ab
I20260808 08:27:38.906181 14519 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:27:38.932603 14519 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:27:38.933576 14519 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:27:38.941998 14519 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1: Generated new TSK 0
I20260808 08:27:38.942663 14519 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.235.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab.
Entry for principal kudu/127.22.235.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab.
I20260808 08:27:38.975601 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.193:0
--local_ip_for_outbound_sockets=127.22.235.193
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:46193
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab
--principal=kudu/127.22.235.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:40861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:27:39.088279 14547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:27:39.088537 14547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:27:39.088578 14547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:27:39.092552 14547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:27:39.092603 14547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:27:39.092679 14547 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.193
I20260808 08:27:39.097870 14547 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:40861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab
--principal=kudu/127.22.235.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:46193
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.193
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:27:39.098840 14547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:27:39.099825 14547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:27:39.109205 14555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:27:39.109190 14553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:27:39.109233 14552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:27:39.109515 14547 server_base.cc:1034] running on GCE node
I20260808 08:27:39.109894 14547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:27:39.110585 14547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:27:39.111747 14547 hybrid_clock.cc:648] HybridClock initialized: now 1786177659111712 us; error 47 us; skew 500 ppm
Aug 08 08:27:39 dist-test-slave-f7th krb5kdc[14368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177659, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:27:39.114370 14547 init.cc:377] Logged in from keytab as kudu/127.22.235.193@KRBTEST.COM (short username kudu)
I20260808 08:27:39.115190 14547 webserver.cc:460] Webserver started at http://127.22.235.193:42437/ using document root <none> and password file <none>
I20260808 08:27:39.115722 14547 fs_manager.cc:359] Metadata directory not provided
I20260808 08:27:39.115788 14547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:27:39.115996 14547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:27:39.117977 14547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-0/data/instance:
uuid: "b2e3f9bc786849869ce8e2432df29b53"
format_stamp: "Formatted at 2026-08-08 08:27:39 on dist-test-slave-f7th"
server_key: "24ded7bead70c8e00ea7681813f84075"
server_key_iv: "7cfda73c74bfc09c77b0c8ec0b029c65"
server_key_version: "encryptionkey@0"
I20260808 08:27:39.118507 14547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance:
uuid: "b2e3f9bc786849869ce8e2432df29b53"
format_stamp: "Formatted at 2026-08-08 08:27:39 on dist-test-slave-f7th"
server_key: "24ded7bead70c8e00ea7681813f84075"
server_key_iv: "7cfda73c74bfc09c77b0c8ec0b029c65"
server_key_version: "encryptionkey@0"
I20260808 08:27:39.122174 14547 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 08:27:39.124393 14567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:39.125591 14547 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:27:39.125695 14547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-0/wal
uuid: "b2e3f9bc786849869ce8e2432df29b53"
format_stamp: "Formatted at 2026-08-08 08:27:39 on dist-test-slave-f7th"
server_key: "24ded7bead70c8e00ea7681813f84075"
server_key_iv: "7cfda73c74bfc09c77b0c8ec0b029c65"
server_key_version: "encryptionkey@0"
I20260808 08:27:39.125785 14547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:27:39.135773 14547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:27:39.138370 14547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:27:39.138537 14547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:27:39.139063 14547 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:27:39.139837 14547 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:27:39.139882 14547 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:39.139933 14547 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:27:39.139962 14547 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:39.148236 14547 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.193:44193
I20260808 08:27:39.148259 14757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.193:44193 every 8 connection(s)
I20260808 08:27:39.149094 14547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
I20260808 08:27:39.151701 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 14547
I20260808 08:27:39.151835 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance
I20260808 08:27:39.152154 23471 external_mini_cluster.cc:1428] Setting key 0ef4fd94875ae2ca248d423239d26a5f
Aug 08 08:27:39 dist-test-slave-f7th krb5kdc[14368](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177659, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.235.194@KRBTEST.COM; defaulting to no policy
I20260808 08:27:39.162990 14510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:39.150804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:33381 (local address 127.22.235.254:46193)
0808 08:27:39.150927 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:39.150931 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:39.151767 (+ 836us) server_negotiation.cc:407] Connection header received
0808 08:27:39.153376 (+ 1609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:39.153380 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:39.153431 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:39.153503 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:27:39.154944 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:39.155493 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:39.156712 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:39.157041 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:39.157150 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:39.159648 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:27:39.159652 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:27:39.159654 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:27:39.159677 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:27:39.161053 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:39.161526 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:39.161529 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:39.161531 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:39.161631 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:39.161950 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:39.161953 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:39.161955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:39.162132 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:27:39.162218 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:39.162690 (+ 472us) server_negotiation.cc:299] Negotiation successful
0808 08:27:39.162819 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 08:27:39.163883 14762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:39.151136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.193:33381)
0808 08:27:39.151608 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:39.151666 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:39.153185 (+ 1519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:39.153683 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:27:39.153692 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:27:39.154153 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:27:39.154790 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:39.154809 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:39.155615 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:39.155619 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:39.156577 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:39.156586 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:39.157172 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:39.157174 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:39.157359 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:39.157905 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:27:39.157925 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:27:39.159493 (+ 1568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:27:39.161158 (+ 1665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:39.161164 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:39.161179 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:39.161432 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:39.161717 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:39.161721 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:39.161723 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:39.161854 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:39.162233 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:27:39.162239 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:27:39.162470 (+ 231us) client_negotiation.cc:769] Sending connection context
0808 08:27:39.162629 (+ 159us) client_negotiation.cc:240] Negotiation successful
0808 08:27:39.162957 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":110,"threads_started":1}
I20260808 08:27:39.165130 14758 heartbeater.cc:347] Connected to a master server at 127.22.235.254:46193
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.194@KRBTEST.COM" created.
I20260808 08:27:39.165365 14758 heartbeater.cc:464] Registering TS with master...
I20260808 08:27:39.165838 14758 heartbeater.cc:511] Master 127.22.235.254:46193 requested a full tablet report, sending...
I20260808 08:27:39.167299 14452 ts_manager.cc:194] Registered new tserver with Master: b2e3f9bc786849869ce8e2432df29b53 (127.22.235.193:44193)
I20260808 08:27:39.168444 14452 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.193@KRBTEST.COM'} at 127.22.235.193:33381
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab.
Entry for principal kudu/127.22.235.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab.
I20260808 08:27:39.184468 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.194:0
--local_ip_for_outbound_sockets=127.22.235.194
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:46193
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab
--principal=kudu/127.22.235.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:40861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:27:39.302178 14767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:27:39.302454 14767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:27:39.302497 14767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:27:39.306773 14767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:27:39.306849 14767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:27:39.306906 14767 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.194
I20260808 08:27:39.312366 14767 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:40861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab
--principal=kudu/127.22.235.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:46193
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14767
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.194
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:27:39.313393 14767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:27:39.314356 14767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:27:39.324082 14781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:27:39.324061 14783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:27:39.324162 14767 server_base.cc:1034] running on GCE node
W20260808 08:27:39.324082 14780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:27:39.324747 14767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:27:39.325412 14767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:27:39.326589 14767 hybrid_clock.cc:648] HybridClock initialized: now 1786177659326566 us; error 29 us; skew 500 ppm
Aug 08 08:27:39 dist-test-slave-f7th krb5kdc[14368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177659, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:27:39.329444 14767 init.cc:377] Logged in from keytab as kudu/127.22.235.194@KRBTEST.COM (short username kudu)
I20260808 08:27:39.330329 14767 webserver.cc:460] Webserver started at http://127.22.235.194:42477/ using document root <none> and password file <none>
I20260808 08:27:39.330879 14767 fs_manager.cc:359] Metadata directory not provided
I20260808 08:27:39.330940 14767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:27:39.331149 14767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:27:39.333189 14767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-1/data/instance:
uuid: "a6d6813abd514f13a3e5880174713c3a"
format_stamp: "Formatted at 2026-08-08 08:27:39 on dist-test-slave-f7th"
server_key: "8305ba8ff1f3802b214318a90262e41f"
server_key_iv: "8dc3c34822d4a9cf8efbcb79d34413c6"
server_key_version: "encryptionkey@0"
I20260808 08:27:39.333725 14767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance:
uuid: "a6d6813abd514f13a3e5880174713c3a"
format_stamp: "Formatted at 2026-08-08 08:27:39 on dist-test-slave-f7th"
server_key: "8305ba8ff1f3802b214318a90262e41f"
server_key_iv: "8dc3c34822d4a9cf8efbcb79d34413c6"
server_key_version: "encryptionkey@0"
I20260808 08:27:39.337517 14767 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 08:27:39.339986 14792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:39.341245 14767 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:27:39.341346 14767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-1/wal
uuid: "a6d6813abd514f13a3e5880174713c3a"
format_stamp: "Formatted at 2026-08-08 08:27:39 on dist-test-slave-f7th"
server_key: "8305ba8ff1f3802b214318a90262e41f"
server_key_iv: "8dc3c34822d4a9cf8efbcb79d34413c6"
server_key_version: "encryptionkey@0"
I20260808 08:27:39.341445 14767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:27:39.350675 14767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:27:39.353602 14767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:27:39.353772 14767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:27:39.354352 14767 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:27:39.355118 14767 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:27:39.355173 14767 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:39.355219 14767 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:27:39.355238 14767 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:39.363722 14767 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.194:42205
I20260808 08:27:39.363799 14985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.194:42205 every 8 connection(s)
I20260808 08:27:39.364647 14767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
I20260808 08:27:39.371031 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 14767
I20260808 08:27:39.371145 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance
I20260808 08:27:39.371399 23471 external_mini_cluster.cc:1428] Setting key a92f90a5dbd9aa010b6932832848ce35
Aug 08 08:27:39 dist-test-slave-f7th krb5kdc[14368](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177659, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.194@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:27:39.378258 14510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:39.366466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:59491 (local address 127.22.235.254:46193)
0808 08:27:39.366589 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:39.366593 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:39.367407 (+ 814us) server_negotiation.cc:407] Connection header received
0808 08:27:39.368801 (+ 1394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:39.368805 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:39.368871 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:39.368957 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:27:39.370102 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:39.370633 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:39.371860 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:39.372169 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:39.372260 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:39.374866 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:27:39.374871 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:27:39.374874 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:27:39.374903 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:27:39.376212 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:39.376770 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:39.376772 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:39.376774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:39.376844 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:39.377183 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:39.377185 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:39.377186 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:39.377349 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:27:39.377418 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:39.378038 (+ 620us) server_negotiation.cc:299] Negotiation successful
0808 08:27:39.378141 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 08:27:39.379381 14989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:39.366799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.194:59491)
0808 08:27:39.367249 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:39.367290 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:39.368555 (+ 1265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:39.369134 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:27:39.369141 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:27:39.369486 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:27:39.369944 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:39.369958 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:39.370756 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:39.370776 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:39.371720 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:39.371733 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:39.372340 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:39.372345 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:39.372531 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:39.373143 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:27:39.373159 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:27:39.374699 (+ 1540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:27:39.376325 (+ 1626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:39.376331 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:39.376346 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:39.376684 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:39.376932 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:39.376935 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:39.376937 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:39.377105 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:39.377437 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:27:39.377444 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:27:39.377767 (+ 323us) client_negotiation.cc:769] Sending connection context
0808 08:27:39.377989 (+ 222us) client_negotiation.cc:240] Negotiation successful
0808 08:27:39.378323 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":103,"threads_started":1}
I20260808 08:27:39.380527 14987 heartbeater.cc:347] Connected to a master server at 127.22.235.254:46193
I20260808 08:27:39.380861 14987 heartbeater.cc:464] Registering TS with master...
I20260808 08:27:39.381474 14987 heartbeater.cc:511] Master 127.22.235.254:46193 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.22.235.195@KRBTEST.COM; defaulting to no policy
I20260808 08:27:39.382666 14452 ts_manager.cc:194] Registered new tserver with Master: a6d6813abd514f13a3e5880174713c3a (127.22.235.194:42205)
I20260808 08:27:39.383276 14452 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.194@KRBTEST.COM'} at 127.22.235.194:59491
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab.
Entry for principal kudu/127.22.235.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab.
I20260808 08:27:39.401438 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.195:0
--local_ip_for_outbound_sockets=127.22.235.195
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:46193
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab
--principal=kudu/127.22.235.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:40861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:27:39.513896 14998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:27:39.514153 14998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:27:39.514194 14998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:27:39.518136 14998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:27:39.518190 14998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:27:39.518244 14998 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.195
I20260808 08:27:39.523290 14998 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:40861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab
--principal=kudu/127.22.235.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:46193
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14998
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.195
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:27:39.524236 14998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:27:39.525214 14998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:27:39.534792 15013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:27:39.534801 15019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:27:39.534801 15015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:27:39.534893 14998 server_base.cc:1034] running on GCE node
I20260808 08:27:39.535382 14998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:27:39.536001 14998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:27:39.537207 14998 hybrid_clock.cc:648] HybridClock initialized: now 1786177659537189 us; error 27 us; skew 500 ppm
Aug 08 08:27:39 dist-test-slave-f7th krb5kdc[14368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177659, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:27:39.539814 14998 init.cc:377] Logged in from keytab as kudu/127.22.235.195@KRBTEST.COM (short username kudu)
I20260808 08:27:39.540659 14998 webserver.cc:460] Webserver started at http://127.22.235.195:43719/ using document root <none> and password file <none>
I20260808 08:27:39.541180 14998 fs_manager.cc:359] Metadata directory not provided
I20260808 08:27:39.541236 14998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:27:39.541400 14998 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:27:39.543473 14998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-2/data/instance:
uuid: "a8037a31939547ffa83bb358dc39307e"
format_stamp: "Formatted at 2026-08-08 08:27:39 on dist-test-slave-f7th"
server_key: "35da105d6b2ff23bd8af75eca5fed51a"
server_key_iv: "2ec23ad4c3b20e937be4c244ffdfe840"
server_key_version: "encryptionkey@0"
I20260808 08:27:39.544087 14998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance:
uuid: "a8037a31939547ffa83bb358dc39307e"
format_stamp: "Formatted at 2026-08-08 08:27:39 on dist-test-slave-f7th"
server_key: "35da105d6b2ff23bd8af75eca5fed51a"
server_key_iv: "2ec23ad4c3b20e937be4c244ffdfe840"
server_key_version: "encryptionkey@0"
I20260808 08:27:39.548163 14998 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 08:27:39.550434 15029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:39.551586 14998 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:27:39.551707 14998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-2/wal
uuid: "a8037a31939547ffa83bb358dc39307e"
format_stamp: "Formatted at 2026-08-08 08:27:39 on dist-test-slave-f7th"
server_key: "35da105d6b2ff23bd8af75eca5fed51a"
server_key_iv: "2ec23ad4c3b20e937be4c244ffdfe840"
server_key_version: "encryptionkey@0"
I20260808 08:27:39.551795 14998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:27:39.561007 14998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:27:39.563840 14998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:27:39.563984 14998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:27:39.564486 14998 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:27:39.565347 14998 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:27:39.565392 14998 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:39.565430 14998 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:27:39.565450 14998 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:39.573968 14998 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.195:37593
I20260808 08:27:39.574005 15225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.195:37593 every 8 connection(s)
I20260808 08:27:39.574818 14998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
I20260808 08:27:39.578086 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 14998
I20260808 08:27:39.578223 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance
I20260808 08:27:39.578511 23471 external_mini_cluster.cc:1428] Setting key 1ff03a774105d811f2855fc68fd4ff30
Aug 08 08:27:39 dist-test-slave-f7th krb5kdc[14368](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177659, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.195@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:27:39.587790 14510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:39.576556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:54589 (local address 127.22.235.254:46193)
0808 08:27:39.576726 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:39.576730 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:39.577512 (+ 782us) server_negotiation.cc:407] Connection header received
0808 08:27:39.579140 (+ 1628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:39.579143 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:39.579197 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:39.579282 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:27:39.580565 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:39.581134 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:39.582234 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:39.582455 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:39.582539 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:39.584811 (+ 2272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:27:39.584814 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:27:39.584816 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:27:39.584838 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:27:39.585933 (+ 1095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:39.586388 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:39.586392 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:39.586394 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:39.586470 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:39.586769 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:39.586772 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:39.586774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:39.586959 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:27:39.587051 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:39.587512 (+ 461us) server_negotiation.cc:299] Negotiation successful
0808 08:27:39.587630 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 08:27:39.588532 15229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:39.576927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.195:54589)
0808 08:27:39.577374 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:39.577414 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:39.578893 (+ 1479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:39.579440 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:27:39.579449 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:27:39.579923 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:27:39.580437 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:39.580456 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:39.581250 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:39.581254 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:39.582123 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:39.582132 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:39.582586 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:39.582590 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:39.582745 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:39.583264 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:27:39.583281 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:27:39.584658 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:27:39.586038 (+ 1380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:39.586044 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:39.586058 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:39.586298 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:39.586550 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:39.586553 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:39.586556 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:39.586683 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:39.587062 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:27:39.587069 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:27:39.587294 (+ 225us) client_negotiation.cc:769] Sending connection context
0808 08:27:39.587452 (+ 158us) client_negotiation.cc:240] Negotiation successful
0808 08:27:39.587764 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":114,"threads_started":1}
I20260808 08:27:39.589570 15226 heartbeater.cc:347] Connected to a master server at 127.22.235.254:46193
I20260808 08:27:39.589792 15226 heartbeater.cc:464] Registering TS with master...
I20260808 08:27:39.590265 15226 heartbeater.cc:511] Master 127.22.235.254:46193 requested a full tablet report, sending...
I20260808 08:27:39.591269 14452 ts_manager.cc:194] Registered new tserver with Master: a8037a31939547ffa83bb358dc39307e (127.22.235.195:37593)
I20260808 08:27:39.591964 14452 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.195@KRBTEST.COM'} at 127.22.235.195:54589
I20260808 08:27:39.592222 23471 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 08:27:39.600682 14510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:39.594183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38676 (local address 127.22.235.254:46193)
0808 08:27:39.594298 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:39.594301 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:39.594500 (+ 199us) server_negotiation.cc:407] Connection header received
0808 08:27:39.594597 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:39.594599 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:39.594635 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:39.594697 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:27:39.595321 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:39.595886 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:39.596789 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:39.596998 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:39.597071 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:39.598261 (+ 1190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:27:39.598264 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:27:39.598266 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:27:39.598287 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:27:39.599394 (+ 1107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:39.599739 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:39.599741 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:39.599742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:39.599809 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:39.600003 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:39.600005 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:39.600006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:39.600192 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:27:39.600288 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:39.600442 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 08:27:39.600540 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 08:27:39.603379 14452 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38676:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 08:27:39.606381 14452 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 08:27:39.620078 15256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:39.614913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.194:42205 (local address 127.0.0.1:58122)
0808 08:27:39.615611 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:39.615625 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:39.615711 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:39.616358 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:27:39.616361 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:27:39.616382 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:27:39.616485 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:39.616492 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:39.617749 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:39.617752 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:39.618681 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:39.618691 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:39.619549 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:39.619553 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:39.619665 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:39.619703 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 08:27:39.619810 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 08:27:39.619929 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"mutex_wait_us":1,"thread_start_us":90,"threads_started":1}
I20260808 08:27:39.620254 15249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:39.614256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.195:37593 (local address 127.0.0.1:45184)
0808 08:27:39.614734 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:39.614746 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:39.614830 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:39.615766 (+ 936us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:27:39.615773 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:27:39.615794 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:27:39.615920 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:39.615927 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:39.617505 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:39.617509 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:39.618849 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:39.618855 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:39.619836 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:39.619839 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:39.619958 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:39.619980 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 08:27:39.620046 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 08:27:39.620110 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"mutex_wait_us":217,"spinlock_wait_cycles":896,"thread_start_us":86,"threads_started":1}
I20260808 08:27:39.620291 15253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:39.614185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:44193 (local address 127.0.0.1:56656)
0808 08:27:39.614651 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:39.614702 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:39.614830 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:39.615936 (+ 1106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:27:39.615939 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:27:39.615956 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:27:39.616109 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:39.616116 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:39.617261 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:39.617264 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:39.618562 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:39.618573 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:39.619800 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:39.619803 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:39.619885 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:39.619968 (+ 83us) client_negotiation.cc:769] Sending connection context
0808 08:27:39.620043 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 08:27:39.620117 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"mutex_wait_us":45,"thread_start_us":81,"threads_started":1}
I20260808 08:27:39.620692 15254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:39.614340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56656 (local address 127.22.235.193:44193)
0808 08:27:39.615159 (+ 819us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:39.615170 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:39.615195 (+ 25us) server_negotiation.cc:407] Connection header received
0808 08:27:39.615384 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:39.615389 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:39.615611 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:39.615761 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:27:39.616253 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:39.617135 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:39.618692 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:39.619492 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:39.619610 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:39.619733 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:39.620450 (+ 717us) server_negotiation.cc:299] Negotiation successful
0808 08:27:39.620539 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":713,"thread_start_us":152,"threads_started":1}
I20260808 08:27:39.620750 15258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:39.614963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45184 (local address 127.22.235.195:37593)
0808 08:27:39.615266 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:39.615278 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:39.615301 (+ 23us) server_negotiation.cc:407] Connection header received
0808 08:27:39.615375 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:39.615380 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:39.615603 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:39.616067 (+ 464us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:27:39.616208 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:39.617373 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:39.618957 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:39.619713 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:39.619826 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:39.619934 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:39.620500 (+ 566us) server_negotiation.cc:299] Negotiation successful
0808 08:27:39.620578 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":112,"threads_started":1}
I20260808 08:27:39.620887 15259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:39.615134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58122 (local address 127.22.235.194:42205)
0808 08:27:39.615949 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:39.615961 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:39.615982 (+ 21us) server_negotiation.cc:407] Connection header received
0808 08:27:39.616066 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:39.616071 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:39.616229 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:39.616373 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:27:39.616595 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:39.617632 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:39.618822 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:39.619433 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:39.619550 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:39.619687 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:39.620022 (+ 335us) server_negotiation.cc:299] Negotiation successful
0808 08:27:39.620114 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":720,"thread_start_us":107,"threads_started":1}
I20260808 08:27:39.623390 14867 tablet_service.cc:1467] Processing CreateTablet for tablet 5378eaa5aaa34fd8a04e01cd8a3d8c1a (DEFAULT_TABLE table=test-table [id=ee9efc8b062f43038e0a6998eecce16d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:27:39.623389 14655 tablet_service.cc:1467] Processing CreateTablet for tablet 5378eaa5aaa34fd8a04e01cd8a3d8c1a (DEFAULT_TABLE table=test-table [id=ee9efc8b062f43038e0a6998eecce16d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:27:39.624346 14655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5378eaa5aaa34fd8a04e01cd8a3d8c1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:27:39.624346 14867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5378eaa5aaa34fd8a04e01cd8a3d8c1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:27:39.630381 15260 tablet_bootstrap.cc:492] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a6d6813abd514f13a3e5880174713c3a: Bootstrap starting.
I20260808 08:27:39.630769 15106 tablet_service.cc:1467] Processing CreateTablet for tablet 5378eaa5aaa34fd8a04e01cd8a3d8c1a (DEFAULT_TABLE table=test-table [id=ee9efc8b062f43038e0a6998eecce16d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:27:39.631523 15261 tablet_bootstrap.cc:492] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P b2e3f9bc786849869ce8e2432df29b53: Bootstrap starting.
I20260808 08:27:39.631835 15106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5378eaa5aaa34fd8a04e01cd8a3d8c1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:27:39.632887 15260 tablet_bootstrap.cc:654] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a6d6813abd514f13a3e5880174713c3a: Neither blocks nor log segments found. Creating new log.
I20260808 08:27:39.633643 15261 tablet_bootstrap.cc:654] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P b2e3f9bc786849869ce8e2432df29b53: Neither blocks nor log segments found. Creating new log.
I20260808 08:27:39.633805 15260 log.cc:824] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a6d6813abd514f13a3e5880174713c3a: Log is configured to *not* fsync() on all Append() calls
I20260808 08:27:39.634948 15261 log.cc:824] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P b2e3f9bc786849869ce8e2432df29b53: Log is configured to *not* fsync() on all Append() calls
I20260808 08:27:39.635753 15260 tablet_bootstrap.cc:492] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a6d6813abd514f13a3e5880174713c3a: No bootstrap required, opened a new log
I20260808 08:27:39.635949 15260 ts_tablet_manager.cc:1397] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a6d6813abd514f13a3e5880174713c3a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 08:27:39.636955 15261 tablet_bootstrap.cc:492] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P b2e3f9bc786849869ce8e2432df29b53: No bootstrap required, opened a new log
I20260808 08:27:39.637140 15261 ts_tablet_manager.cc:1397] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P b2e3f9bc786849869ce8e2432df29b53: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 08:27:39.638957 15260 raft_consensus.cc:348] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a6d6813abd514f13a3e5880174713c3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2e3f9bc786849869ce8e2432df29b53" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44193 } } peers { permanent_uuid: "a8037a31939547ffa83bb358dc39307e" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37593 } } peers { permanent_uuid: "a6d6813abd514f13a3e5880174713c3a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 42205 } }
I20260808 08:27:39.639222 15260 raft_consensus.cc:374] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a6d6813abd514f13a3e5880174713c3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:27:39.639277 15260 raft_consensus.cc:729] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a6d6813abd514f13a3e5880174713c3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6d6813abd514f13a3e5880174713c3a, State: Initialized, Role: FOLLOWER
I20260808 08:27:39.639704 15265 tablet_bootstrap.cc:492] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a8037a31939547ffa83bb358dc39307e: Bootstrap starting.
I20260808 08:27:39.639679 15260 consensus_queue.cc:260] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a6d6813abd514f13a3e5880174713c3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2e3f9bc786849869ce8e2432df29b53" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44193 } } peers { permanent_uuid: "a8037a31939547ffa83bb358dc39307e" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37593 } } peers { permanent_uuid: "a6d6813abd514f13a3e5880174713c3a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 42205 } }
I20260808 08:27:39.640287 14987 heartbeater.cc:503] Master 127.22.235.254:46193 was elected leader, sending a full tablet report...
I20260808 08:27:39.640713 15260 ts_tablet_manager.cc:1428] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a6d6813abd514f13a3e5880174713c3a: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 08:27:39.641067 15261 raft_consensus.cc:348] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P b2e3f9bc786849869ce8e2432df29b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2e3f9bc786849869ce8e2432df29b53" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44193 } } peers { permanent_uuid: "a8037a31939547ffa83bb358dc39307e" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37593 } } peers { permanent_uuid: "a6d6813abd514f13a3e5880174713c3a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 42205 } }
I20260808 08:27:39.641302 15261 raft_consensus.cc:374] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P b2e3f9bc786849869ce8e2432df29b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:27:39.641356 15261 raft_consensus.cc:729] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P b2e3f9bc786849869ce8e2432df29b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e3f9bc786849869ce8e2432df29b53, State: Initialized, Role: FOLLOWER
I20260808 08:27:39.641808 15261 consensus_queue.cc:260] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P b2e3f9bc786849869ce8e2432df29b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2e3f9bc786849869ce8e2432df29b53" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44193 } } peers { permanent_uuid: "a8037a31939547ffa83bb358dc39307e" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37593 } } peers { permanent_uuid: "a6d6813abd514f13a3e5880174713c3a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 42205 } }
I20260808 08:27:39.641942 15265 tablet_bootstrap.cc:654] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a8037a31939547ffa83bb358dc39307e: Neither blocks nor log segments found. Creating new log.
I20260808 08:27:39.642488 14758 heartbeater.cc:503] Master 127.22.235.254:46193 was elected leader, sending a full tablet report...
I20260808 08:27:39.642810 15261 ts_tablet_manager.cc:1428] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P b2e3f9bc786849869ce8e2432df29b53: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 08:27:39.643200 15265 log.cc:824] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a8037a31939547ffa83bb358dc39307e: Log is configured to *not* fsync() on all Append() calls
I20260808 08:27:39.645260 15265 tablet_bootstrap.cc:492] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a8037a31939547ffa83bb358dc39307e: No bootstrap required, opened a new log
I20260808 08:27:39.645457 15265 ts_tablet_manager.cc:1397] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a8037a31939547ffa83bb358dc39307e: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 08:27:39.648283 15265 raft_consensus.cc:348] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a8037a31939547ffa83bb358dc39307e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2e3f9bc786849869ce8e2432df29b53" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44193 } } peers { permanent_uuid: "a8037a31939547ffa83bb358dc39307e" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37593 } } peers { permanent_uuid: "a6d6813abd514f13a3e5880174713c3a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 42205 } }
I20260808 08:27:39.648478 15265 raft_consensus.cc:374] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a8037a31939547ffa83bb358dc39307e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:27:39.648535 15265 raft_consensus.cc:729] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a8037a31939547ffa83bb358dc39307e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8037a31939547ffa83bb358dc39307e, State: Initialized, Role: FOLLOWER
I20260808 08:27:39.648964 15265 consensus_queue.cc:260] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a8037a31939547ffa83bb358dc39307e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2e3f9bc786849869ce8e2432df29b53" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44193 } } peers { permanent_uuid: "a8037a31939547ffa83bb358dc39307e" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37593 } } peers { permanent_uuid: "a6d6813abd514f13a3e5880174713c3a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 42205 } }
I20260808 08:27:39.649549 15226 heartbeater.cc:503] Master 127.22.235.254:46193 was elected leader, sending a full tablet report...
I20260808 08:27:39.649832 15265 ts_tablet_manager.cc:1428] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a8037a31939547ffa83bb358dc39307e: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
W20260808 08:27:39.650214 14759 tablet.cc:2369] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P b2e3f9bc786849869ce8e2432df29b53: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 08:27:39.825773 15228 tablet.cc:2369] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a8037a31939547ffa83bb358dc39307e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 08:27:39.865836 14988 tablet.cc:2369] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a6d6813abd514f13a3e5880174713c3a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 08:27:40.037878 15267 raft_consensus.cc:482] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a6d6813abd514f13a3e5880174713c3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 08:27:40.038108 15267 raft_consensus.cc:504] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a6d6813abd514f13a3e5880174713c3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2e3f9bc786849869ce8e2432df29b53" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44193 } } peers { permanent_uuid: "a8037a31939547ffa83bb358dc39307e" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37593 } } peers { permanent_uuid: "a6d6813abd514f13a3e5880174713c3a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 42205 } }
I20260808 08:27:40.039239 15267 leader_election.cc:288] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a6d6813abd514f13a3e5880174713c3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b2e3f9bc786849869ce8e2432df29b53 (127.22.235.193:44193), a8037a31939547ffa83bb358dc39307e (127.22.235.195:37593)
I20260808 08:27:40.042886 15269 raft_consensus.cc:482] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P b2e3f9bc786849869ce8e2432df29b53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 08:27:40.043169 15276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:40.039515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:44193 (local address 127.22.235.194:41303)
0808 08:27:40.039964 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:40.039978 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:40.040105 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:40.040348 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:27:40.040351 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:27:40.040365 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:27:40.040498 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:40.040505 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:40.041212 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:40.041215 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:40.042085 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:40.042094 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:40.042721 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:40.042724 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:40.042830 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:40.042856 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 08:27:40.042936 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 08:27:40.043010 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"mutex_wait_us":88,"thread_start_us":98,"threads_started":1}
I20260808 08:27:40.043171 15254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:40.039646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:41303 (local address 127.22.235.193:44193)
0808 08:27:40.039799 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:40.039804 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:40.040086 (+ 282us) server_negotiation.cc:407] Connection header received
0808 08:27:40.040194 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:40.040197 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:40.040251 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:40.040327 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:27:40.040606 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:40.041108 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:40.042204 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:40.042610 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:40.042717 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:40.042774 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:40.042969 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 08:27:40.043054 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 08:27:40.043186 15258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:40.039812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:34957 (local address 127.22.235.195:37593)
0808 08:27:40.039934 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:40.039938 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:40.039955 (+ 17us) server_negotiation.cc:407] Connection header received
0808 08:27:40.040091 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:40.040095 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:40.040141 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:40.040215 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:27:40.040549 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:40.041100 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:40.042221 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:40.042630 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:40.042745 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:40.042796 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:40.043027 (+ 231us) server_negotiation.cc:299] Negotiation successful
0808 08:27:40.043091 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 08:27:40.043130 15269 raft_consensus.cc:504] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P b2e3f9bc786849869ce8e2432df29b53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2e3f9bc786849869ce8e2432df29b53" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44193 } } peers { permanent_uuid: "a8037a31939547ffa83bb358dc39307e" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37593 } } peers { permanent_uuid: "a6d6813abd514f13a3e5880174713c3a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 42205 } }
I20260808 08:27:40.043665 15274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:40.039515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.195:37593 (local address 127.22.235.194:34957)
0808 08:27:40.039870 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:40.039886 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:40.039988 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:40.040233 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:27:40.040236 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:27:40.040253 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:27:40.040425 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:40.040434 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:40.041210 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:40.041214 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:40.042094 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:40.042104 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:40.042778 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:40.042781 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:40.042890 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:40.042909 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 08:27:40.043457 (+ 548us) client_negotiation.cc:240] Negotiation successful
0808 08:27:40.043535 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"mutex_wait_us":74,"thread_start_us":70,"threads_started":1}
I20260808 08:27:40.043812 14683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5378eaa5aaa34fd8a04e01cd8a3d8c1a" candidate_uuid: "a6d6813abd514f13a3e5880174713c3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e3f9bc786849869ce8e2432df29b53" is_pre_election: true
I20260808 08:27:40.044072 14683 raft_consensus.cc:2454] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P b2e3f9bc786849869ce8e2432df29b53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6d6813abd514f13a3e5880174713c3a in term 0.
I20260808 08:27:40.044312 15143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5378eaa5aaa34fd8a04e01cd8a3d8c1a" candidate_uuid: "a6d6813abd514f13a3e5880174713c3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8037a31939547ffa83bb358dc39307e" is_pre_election: true
I20260808 08:27:40.044601 14798 leader_election.cc:302] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a6d6813abd514f13a3e5880174713c3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6d6813abd514f13a3e5880174713c3a, b2e3f9bc786849869ce8e2432df29b53; no voters:
I20260808 08:27:40.044652 15143 raft_consensus.cc:2454] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a8037a31939547ffa83bb358dc39307e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6d6813abd514f13a3e5880174713c3a in term 0.
I20260808 08:27:40.044922 15267 raft_consensus.cc:2781] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a6d6813abd514f13a3e5880174713c3a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 08:27:40.045027 15267 raft_consensus.cc:482] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a6d6813abd514f13a3e5880174713c3a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 08:27:40.045082 15267 raft_consensus.cc:3037] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a6d6813abd514f13a3e5880174713c3a [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:27:40.045286 15269 leader_election.cc:288] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P b2e3f9bc786849869ce8e2432df29b53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8037a31939547ffa83bb358dc39307e (127.22.235.195:37593), a6d6813abd514f13a3e5880174713c3a (127.22.235.194:42205)
I20260808 08:27:40.046315 15267 raft_consensus.cc:504] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a6d6813abd514f13a3e5880174713c3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2e3f9bc786849869ce8e2432df29b53" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44193 } } peers { permanent_uuid: "a8037a31939547ffa83bb358dc39307e" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37593 } } peers { permanent_uuid: "a6d6813abd514f13a3e5880174713c3a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 42205 } }
I20260808 08:27:40.046610 15267 leader_election.cc:288] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a6d6813abd514f13a3e5880174713c3a [CANDIDATE]: Term 1 election: Requested vote from peers b2e3f9bc786849869ce8e2432df29b53 (127.22.235.193:44193), a8037a31939547ffa83bb358dc39307e (127.22.235.195:37593)
I20260808 08:27:40.046912 14683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5378eaa5aaa34fd8a04e01cd8a3d8c1a" candidate_uuid: "a6d6813abd514f13a3e5880174713c3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e3f9bc786849869ce8e2432df29b53"
I20260808 08:27:40.046949 15143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5378eaa5aaa34fd8a04e01cd8a3d8c1a" candidate_uuid: "a6d6813abd514f13a3e5880174713c3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8037a31939547ffa83bb358dc39307e"
I20260808 08:27:40.047058 14683 raft_consensus.cc:3037] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P b2e3f9bc786849869ce8e2432df29b53 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:27:40.047057 15143 raft_consensus.cc:3037] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a8037a31939547ffa83bb358dc39307e [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:27:40.048244 14683 raft_consensus.cc:2454] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P b2e3f9bc786849869ce8e2432df29b53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6d6813abd514f13a3e5880174713c3a in term 1.
I20260808 08:27:40.048245 15143 raft_consensus.cc:2454] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a8037a31939547ffa83bb358dc39307e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6d6813abd514f13a3e5880174713c3a in term 1.
I20260808 08:27:40.048542 14797 leader_election.cc:302] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a6d6813abd514f13a3e5880174713c3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6d6813abd514f13a3e5880174713c3a, a8037a31939547ffa83bb358dc39307e; no voters:
I20260808 08:27:40.048730 15267 raft_consensus.cc:2781] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a6d6813abd514f13a3e5880174713c3a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:27:40.048976 15267 raft_consensus.cc:686] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a6d6813abd514f13a3e5880174713c3a [term 1 LEADER]: Becoming Leader. State: Replica: a6d6813abd514f13a3e5880174713c3a, State: Running, Role: LEADER
I20260808 08:27:40.049310 15267 consensus_queue.cc:237] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a6d6813abd514f13a3e5880174713c3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2e3f9bc786849869ce8e2432df29b53" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44193 } } peers { permanent_uuid: "a8037a31939547ffa83bb358dc39307e" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37593 } } peers { permanent_uuid: "a6d6813abd514f13a3e5880174713c3a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 42205 } }
I20260808 08:27:40.050308 15259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:40.045560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:45947 (local address 127.22.235.194:42205)
0808 08:27:40.045692 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:40.045697 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:40.047004 (+ 1307us) server_negotiation.cc:407] Connection header received
0808 08:27:40.047165 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:40.047169 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:40.047208 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:40.047249 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:27:40.047529 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:40.047967 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:40.049217 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:40.049684 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:40.049785 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:40.049831 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:40.050147 (+ 316us) server_negotiation.cc:299] Negotiation successful
0808 08:27:40.050211 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 08:27:40.050525 15279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:40.045523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.194:42205 (local address 127.22.235.193:45947)
0808 08:27:40.046913 (+ 1390us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:40.046926 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:40.047064 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:40.047341 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:27:40.047343 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:27:40.047356 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:27:40.047431 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:40.047434 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:40.048062 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:40.048066 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:40.049104 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:40.049115 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:40.049786 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:40.049789 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:40.049913 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:40.049935 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 08:27:40.050195 (+ 260us) client_negotiation.cc:240] Negotiation successful
0808 08:27:40.050262 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1332,"mutex_wait_us":1,"thread_start_us":111,"threads_started":1}
I20260808 08:27:40.051004 14909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5378eaa5aaa34fd8a04e01cd8a3d8c1a" candidate_uuid: "b2e3f9bc786849869ce8e2432df29b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6d6813abd514f13a3e5880174713c3a" is_pre_election: true
I20260808 08:27:40.051967 15258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:40.046813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:58853 (local address 127.22.235.195:37593)
0808 08:27:40.046914 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:40.046918 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:40.046930 (+ 12us) server_negotiation.cc:407] Connection header received
0808 08:27:40.046972 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:40.046975 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:40.047016 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:40.047078 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:27:40.048798 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:40.049436 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:40.050997 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:40.051374 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:40.051440 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:40.051473 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:40.051798 (+ 325us) server_negotiation.cc:299] Negotiation successful
0808 08:27:40.051866 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 08:27:40.051973 15278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:40.045418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.195:37593 (local address 127.22.235.193:58853)
0808 08:27:40.045738 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:40.045752 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:40.045830 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:40.048523 (+ 2693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:27:40.048526 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:27:40.048545 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:27:40.048666 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:40.048673 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:40.049576 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:40.049579 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:40.050845 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:40.050857 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:40.051556 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:40.051560 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:40.051651 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:40.051674 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 08:27:40.051770 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 08:27:40.051831 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"mutex_wait_us":193,"thread_start_us":109,"threads_started":1}
I20260808 08:27:40.052397 15143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5378eaa5aaa34fd8a04e01cd8a3d8c1a" candidate_uuid: "b2e3f9bc786849869ce8e2432df29b53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8037a31939547ffa83bb358dc39307e" is_pre_election: true
I20260808 08:27:40.052573 15143 raft_consensus.cc:2379] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a8037a31939547ffa83bb358dc39307e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2e3f9bc786849869ce8e2432df29b53 in current term 1: Already voted for candidate a6d6813abd514f13a3e5880174713c3a in this term.
I20260808 08:27:40.052443 14452 catalog_manager.cc:5495] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a6d6813abd514f13a3e5880174713c3a reported cstate change: term changed from 0 to 1, leader changed from <none> to a6d6813abd514f13a3e5880174713c3a (127.22.235.194). New cstate: current_term: 1 leader_uuid: "a6d6813abd514f13a3e5880174713c3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2e3f9bc786849869ce8e2432df29b53" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44193 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8037a31939547ffa83bb358dc39307e" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6d6813abd514f13a3e5880174713c3a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 42205 } health_report { overall_health: HEALTHY } } }
I20260808 08:27:40.053040 14572 leader_election.cc:302] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P b2e3f9bc786849869ce8e2432df29b53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e3f9bc786849869ce8e2432df29b53; no voters: a6d6813abd514f13a3e5880174713c3a, a8037a31939547ffa83bb358dc39307e
I20260808 08:27:40.053297 15269 raft_consensus.cc:2726] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P b2e3f9bc786849869ce8e2432df29b53 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 08:27:40.070628 15259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:40.067611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58138 (local address 127.22.235.194:42205)
0808 08:27:40.067723 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:40.067727 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:40.067739 (+ 12us) server_negotiation.cc:407] Connection header received
0808 08:27:40.067821 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:40.067823 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:40.067872 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:40.067949 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:27:40.068136 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:40.068609 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:40.069395 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:40.069611 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:40.069683 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:40.069901 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:27:40.070308 (+ 407us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 08:27:40.070396 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:40.070456 (+ 60us) server_negotiation.cc:299] Negotiation successful
0808 08:27:40.070509 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 08:27:40.076148 15143 raft_consensus.cc:1261] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a8037a31939547ffa83bb358dc39307e [term 1 FOLLOWER]: Refusing update from remote peer a6d6813abd514f13a3e5880174713c3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:27:40.076184 14683 raft_consensus.cc:1261] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P b2e3f9bc786849869ce8e2432df29b53 [term 1 FOLLOWER]: Refusing update from remote peer a6d6813abd514f13a3e5880174713c3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:27:40.076783 15267 consensus_queue.cc:1035] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a6d6813abd514f13a3e5880174713c3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8037a31939547ffa83bb358dc39307e" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:27:40.076972 15280 consensus_queue.cc:1035] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a6d6813abd514f13a3e5880174713c3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e3f9bc786849869ce8e2432df29b53" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:27:40.083231 15285 mvcc.cc:205] Tried to move back new op lower bound from 7316183695665135616 to 7316183695564959744. Current Snapshot: MvccSnapshot[applied={T|T < 7316183695665135616}]
I20260808 08:27:40.084407 15287 mvcc.cc:205] Tried to move back new op lower bound from 7316183695665135616 to 7316183695564959744. Current Snapshot: MvccSnapshot[applied={T|T < 7316183695665135616}]
I20260808 08:27:40.091305 14452 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38676:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:27:40.091449 14452 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38676:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:27:40.093768 14452 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1: Sending DeleteTablet for 3 replicas of tablet 5378eaa5aaa34fd8a04e01cd8a3d8c1a
I20260808 08:27:40.094471 15106 tablet_service.cc:1514] Processing DeleteTablet for tablet 5378eaa5aaa34fd8a04e01cd8a3d8c1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:27:40 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:45184
I20260808 08:27:40.094455 14867 tablet_service.cc:1514] Processing DeleteTablet for tablet 5378eaa5aaa34fd8a04e01cd8a3d8c1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:27:40 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:58122
I20260808 08:27:40.094457 14655 tablet_service.cc:1514] Processing DeleteTablet for tablet 5378eaa5aaa34fd8a04e01cd8a3d8c1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:27:40 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:56656
I20260808 08:27:40.094869 15298 tablet_replica.cc:330] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a8037a31939547ffa83bb358dc39307e: stopping tablet replica
I20260808 08:27:40.095100 15300 tablet_replica.cc:330] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a6d6813abd514f13a3e5880174713c3a: stopping tablet replica
I20260808 08:27:40.095351 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 14547
I20260808 08:27:40.095341 15298 raft_consensus.cc:2229] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a8037a31939547ffa83bb358dc39307e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 08:27:40.095377 15300 raft_consensus.cc:2229] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a6d6813abd514f13a3e5880174713c3a [term 1 LEADER]: Raft consensus shutting down.
I20260808 08:27:40.095717 15300 raft_consensus.cc:2258] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a6d6813abd514f13a3e5880174713c3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:27:40.095871 15298 raft_consensus.cc:2258] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a8037a31939547ffa83bb358dc39307e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:27:40.105010 15300 ts_tablet_manager.cc:1905] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a6d6813abd514f13a3e5880174713c3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:27:40.107916 15298 ts_tablet_manager.cc:1905] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a8037a31939547ffa83bb358dc39307e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:27:40.108146 15300 ts_tablet_manager.cc:1918] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a6d6813abd514f13a3e5880174713c3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 08:27:40.108253 15300 log.cc:1196] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a6d6813abd514f13a3e5880174713c3a: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-1/wal/wals/5378eaa5aaa34fd8a04e01cd8a3d8c1a
I20260808 08:27:40.108557 15300 ts_tablet_manager.cc:1939] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a6d6813abd514f13a3e5880174713c3a: Deleting consensus metadata
I20260808 08:27:40.110000 14430 catalog_manager.cc:4841] TS a6d6813abd514f13a3e5880174713c3a (127.22.235.194:42205): tablet 5378eaa5aaa34fd8a04e01cd8a3d8c1a (table test-table [id=ee9efc8b062f43038e0a6998eecce16d]) successfully deleted
I20260808 08:27:40.110534 15298 ts_tablet_manager.cc:1918] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a8037a31939547ffa83bb358dc39307e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 08:27:40.110635 15298 log.cc:1196] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a8037a31939547ffa83bb358dc39307e: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-2/wal/wals/5378eaa5aaa34fd8a04e01cd8a3d8c1a
I20260808 08:27:40.110916 15298 ts_tablet_manager.cc:1939] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P a8037a31939547ffa83bb358dc39307e: Deleting consensus metadata
I20260808 08:27:40.112126 14425 catalog_manager.cc:4841] TS a8037a31939547ffa83bb358dc39307e (127.22.235.195:37593): tablet 5378eaa5aaa34fd8a04e01cd8a3d8c1a (table test-table [id=ee9efc8b062f43038e0a6998eecce16d]) successfully deleted
W20260808 08:27:40.117633 14429 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.235.193:44193 (error 108)
I20260808 08:27:40.117873 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 14767
I20260808 08:27:40.120787 15249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:40.120451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:44193 (local address 127.0.0.1:56668)
0808 08:27:40.120649 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:40.120710 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.193:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
W20260808 08:27:40.120923 14429 catalog_manager.cc:4584] TS b2e3f9bc786849869ce8e2432df29b53 (127.22.235.193:44193): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5378eaa5aaa34fd8a04e01cd8a3d8c1a: Network error: Client connection negotiation failed: client connection to 127.22.235.193:44193: connect: Connection refused (error 111)
I20260808 08:27:40.128928 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 14998
I20260808 08:27:40.138792 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 14391
I20260808 08:27:40.146575 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:46193
--webserver_interface=127.22.235.254
--webserver_port=33717
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:40861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:46193
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:27:40.256112 15305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:27:40.256361 15305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:27:40.256399 15305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:27:40.260242 15305 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:27:40.260293 15305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:27:40.260310 15305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:27:40.260325 15305 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:27:40.260344 15305 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:27:40.265357 15305 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:40861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:46193
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:46193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.235.254
--webserver_port=33717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:27:40.266338 15305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:27:40.267305 15305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:27:40.281682 15314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:27:40.281731 15316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:27:40.281800 15305 server_base.cc:1034] running on GCE node
W20260808 08:27:40.281695 15320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:27:40.282617 15305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:27:40.283785 15305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:27:40.284937 15305 hybrid_clock.cc:648] HybridClock initialized: now 1786177660284925 us; error 32 us; skew 500 ppm
Aug 08 08:27:40 dist-test-slave-f7th krb5kdc[14368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177660, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:27:40.287647 15305 init.cc:377] Logged in from keytab as kudu/127.22.235.254@KRBTEST.COM (short username kudu)
I20260808 08:27:40.288558 15305 webserver.cc:460] Webserver started at http://127.22.235.254:33717/ using document root <none> and password file <none>
I20260808 08:27:40.289146 15305 fs_manager.cc:359] Metadata directory not provided
I20260808 08:27:40.289227 15305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:27:40.293058 15305 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 08:27:40.295193 15333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:40.296365 15305 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:27:40.296476 15305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "44f372ca2df24ab08e3f6ffa192eb3e1"
format_stamp: "Formatted at 2026-08-08 08:27:38 on dist-test-slave-f7th"
server_key: "10dbbed96ad8730cc7fd8ed49f6c1c7d"
server_key_iv: "1682069a614bb82e2ee2706e29085580"
server_key_version: "encryptionkey@0"
I20260808 08:27:40.297137 15305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:27:40.307631 15305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:27:40.310254 15305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:27:40.310434 15305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:27:40.317342 15305 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.254:46193
I20260808 08:27:40.317612 15422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.254:46193 every 8 connection(s)
I20260808 08:27:40.318405 15305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:27:40.322577 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 15305
I20260808 08:27:40.323062 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.193:44193
--local_ip_for_outbound_sockets=127.22.235.193
--tserver_master_addrs=127.22.235.254:46193
--webserver_port=42437
--webserver_interface=127.22.235.193
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab
--principal=kudu/127.22.235.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:40861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 08:27:40.323274 15423 sys_catalog.cc:263] Verifying existing consensus state
I20260808 08:27:40.325393 15423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1: Bootstrap starting.
I20260808 08:27:40.342540 15423 log.cc:824] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1: Log is configured to *not* fsync() on all Append() calls
I20260808 08:27:40.351612 15423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 08:27:40.351964 15423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1: Bootstrap complete.
I20260808 08:27:40.355890 15423 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44f372ca2df24ab08e3f6ffa192eb3e1" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46193 } }
I20260808 08:27:40.356268 15423 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44f372ca2df24ab08e3f6ffa192eb3e1, State: Initialized, Role: FOLLOWER
I20260808 08:27:40.356777 15423 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44f372ca2df24ab08e3f6ffa192eb3e1" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46193 } }
I20260808 08:27:40.356906 15423 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:27:40.356976 15423 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:27:40.357075 15423 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [term 1 FOLLOWER]: Advancing to term 2
I20260808 08:27:40.359025 15423 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44f372ca2df24ab08e3f6ffa192eb3e1" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46193 } }
I20260808 08:27:40.359364 15423 leader_election.cc:302] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44f372ca2df24ab08e3f6ffa192eb3e1; no voters:
I20260808 08:27:40.359653 15423 leader_election.cc:288] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 08:27:40.359750 15428 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 08:27:40.359974 15428 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [term 2 LEADER]: Becoming Leader. State: Replica: 44f372ca2df24ab08e3f6ffa192eb3e1, State: Running, Role: LEADER
I20260808 08:27:40.360270 15428 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44f372ca2df24ab08e3f6ffa192eb3e1" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46193 } }
I20260808 08:27:40.360996 15423 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:27:40.361227 15428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44f372ca2df24ab08e3f6ffa192eb3e1. Latest consensus state: current_term: 2 leader_uuid: "44f372ca2df24ab08e3f6ffa192eb3e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44f372ca2df24ab08e3f6ffa192eb3e1" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46193 } } }
I20260808 08:27:40.361346 15428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [sys.catalog]: This master's current role is: LEADER
I20260808 08:27:40.361764 15436 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:27:40.361737 15430 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "44f372ca2df24ab08e3f6ffa192eb3e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44f372ca2df24ab08e3f6ffa192eb3e1" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46193 } } }
I20260808 08:27:40.361943 15430 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [sys.catalog]: This master's current role is: LEADER
I20260808 08:27:40.366535 15436 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:27:40.367214 15436 catalog_manager.cc:1259] Loaded cluster ID: bd2ab53661ea4a199c69e609adbf39ab
I20260808 08:27:40.367276 15436 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:27:40.368705 15436 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260808 08:27:40.373448 15436 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260808 08:27:40.378772 15436 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:27:40.379494 15436 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1: Loaded TSK: 0
I20260808 08:27:40.380718 15436 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 08:27:40.437984 15425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:27:40.438235 15425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:27:40.438297 15425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:27:40.442252 15425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:27:40.442309 15425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:27:40.442387 15425 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.193
I20260808 08:27:40.447535 15425 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:40861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab
--principal=kudu/127.22.235.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.193:44193
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.235.193
--webserver_port=42437
--tserver_master_addrs=127.22.235.254:46193
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.193
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:27:40.448539 15425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:27:40.449568 15425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:27:40.458776 15462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:27:40.458865 15465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:27:40.458791 15463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:27:40.459300 15425 server_base.cc:1034] running on GCE node
I20260808 08:27:40.459693 15425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:27:40.460317 15425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:27:40.461484 15425 hybrid_clock.cc:648] HybridClock initialized: now 1786177660461432 us; error 65 us; skew 500 ppm
Aug 08 08:27:40 dist-test-slave-f7th krb5kdc[14368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177660, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:27:40.464841 15425 init.cc:377] Logged in from keytab as kudu/127.22.235.193@KRBTEST.COM (short username kudu)
I20260808 08:27:40.465932 15425 webserver.cc:460] Webserver started at http://127.22.235.193:42437/ using document root <none> and password file <none>
I20260808 08:27:40.466634 15425 fs_manager.cc:359] Metadata directory not provided
I20260808 08:27:40.466746 15425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:27:40.471977 15425 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 08:27:40.474746 15475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:40.476212 15425 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 08:27:40.476352 15425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-0/wal
uuid: "b2e3f9bc786849869ce8e2432df29b53"
format_stamp: "Formatted at 2026-08-08 08:27:39 on dist-test-slave-f7th"
server_key: "24ded7bead70c8e00ea7681813f84075"
server_key_iv: "7cfda73c74bfc09c77b0c8ec0b029c65"
server_key_version: "encryptionkey@0"
I20260808 08:27:40.477084 15425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:27:40.503142 15425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:27:40.506987 15425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:27:40.507180 15425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:27:40.507910 15425 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:27:40.509263 15484 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 08:27:40.513608 15425 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 08:27:40.513680 15425 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260808 08:27:40.513742 15425 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 08:27:40.516059 15425 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 08:27:40.516117 15425 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260808 08:27:40.516389 15484 tablet_bootstrap.cc:492] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P b2e3f9bc786849869ce8e2432df29b53: Bootstrap starting.
I20260808 08:27:40.525882 15425 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.193:44193
I20260808 08:27:40.525931 15668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.193:44193 every 8 connection(s)
I20260808 08:27:40.526834 15425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
I20260808 08:27:40.530467 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 15425
I20260808 08:27:40.530875 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.194:42205
--local_ip_for_outbound_sockets=127.22.235.194
--tserver_master_addrs=127.22.235.254:46193
--webserver_port=42477
--webserver_interface=127.22.235.194
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab
--principal=kudu/127.22.235.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:40861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 08:27:40.534518 15484 log.cc:824] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P b2e3f9bc786849869ce8e2432df29b53: Log is configured to *not* fsync() on all Append() calls
Aug 08 08:27:40 dist-test-slave-f7th krb5kdc[14368](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177660, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:27:40.543462 15484 tablet_bootstrap.cc:492] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P b2e3f9bc786849869ce8e2432df29b53: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 08:27:40.543871 15672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:40.529168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:36851 (local address 127.22.235.254:46193)
0808 08:27:40.529596 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:40.529616 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:40.530022 (+ 406us) server_negotiation.cc:407] Connection header received
0808 08:27:40.531542 (+ 1520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:40.531558 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:40.531946 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:40.532301 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:27:40.533823 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:40.535087 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:40.536224 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:40.536763 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:40.536913 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:40.539180 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:27:40.539185 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:27:40.539197 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:27:40.539226 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:27:40.540797 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:40.541276 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:40.541281 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:40.541285 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:40.541394 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:40.541709 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:40.541713 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:40.541714 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:40.542071 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:27:40.542238 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:40.542747 (+ 509us) server_negotiation.cc:299] Negotiation successful
0808 08:27:40.543017 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":149,"threads_started":1}
I20260808 08:27:40.543885 15673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:40.529167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.193:36851)
0808 08:27:40.529830 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:40.529891 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:40.531045 (+ 1154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:40.532442 (+ 1397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:27:40.532452 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:27:40.532998 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:27:40.533619 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:40.533638 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:40.535254 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:40.535258 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:40.536104 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:40.536114 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:40.536897 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:40.536902 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:40.537088 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:40.537619 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:27:40.537636 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:27:40.539004 (+ 1368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:27:40.540905 (+ 1901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:40.540911 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:40.540924 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:40.541170 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:40.541481 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:40.541484 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:40.541486 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:40.541625 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:40.542220 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:27:40.542226 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:27:40.542452 (+ 226us) client_negotiation.cc:769] Sending connection context
0808 08:27:40.542616 (+ 164us) client_negotiation.cc:240] Negotiation successful
0808 08:27:40.542948 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":133,"threads_started":1}
I20260808 08:27:40.544247 15484 tablet_bootstrap.cc:492] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P b2e3f9bc786849869ce8e2432df29b53: Bootstrap complete.
I20260808 08:27:40.545746 15484 ts_tablet_manager.cc:1397] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P b2e3f9bc786849869ce8e2432df29b53: Time spent bootstrapping tablet: real 0.030s user 0.021s sys 0.007s
I20260808 08:27:40.547367 15669 heartbeater.cc:347] Connected to a master server at 127.22.235.254:46193
I20260808 08:27:40.547607 15669 heartbeater.cc:464] Registering TS with master...
I20260808 08:27:40.548153 15669 heartbeater.cc:511] Master 127.22.235.254:46193 requested a full tablet report, sending...
I20260808 08:27:40.548763 15484 raft_consensus.cc:348] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P b2e3f9bc786849869ce8e2432df29b53 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2e3f9bc786849869ce8e2432df29b53" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44193 } } peers { permanent_uuid: "a8037a31939547ffa83bb358dc39307e" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37593 } } peers { permanent_uuid: "a6d6813abd514f13a3e5880174713c3a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 42205 } }
I20260808 08:27:40.549268 15484 raft_consensus.cc:729] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P b2e3f9bc786849869ce8e2432df29b53 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e3f9bc786849869ce8e2432df29b53, State: Initialized, Role: FOLLOWER
I20260808 08:27:40.549749 15484 consensus_queue.cc:260] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P b2e3f9bc786849869ce8e2432df29b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2e3f9bc786849869ce8e2432df29b53" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44193 } } peers { permanent_uuid: "a8037a31939547ffa83bb358dc39307e" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37593 } } peers { permanent_uuid: "a6d6813abd514f13a3e5880174713c3a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 42205 } }
I20260808 08:27:40.550733 15484 ts_tablet_manager.cc:1428] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P b2e3f9bc786849869ce8e2432df29b53: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 08:27:40.550804 15367 ts_manager.cc:194] Registered new tserver with Master: b2e3f9bc786849869ce8e2432df29b53 (127.22.235.193:44193)
I20260808 08:27:40.554117 15367 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.193@KRBTEST.COM'} at 127.22.235.193:36851
I20260808 08:27:40.556520 15669 heartbeater.cc:503] Master 127.22.235.254:46193 was elected leader, sending a full tablet report...
Aug 08 08:27:40 dist-test-slave-f7th krb5kdc[14368](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177660, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for kudu/127.22.235.193@KRBTEST.COM
I20260808 08:27:40.563746 15677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:40.553485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:44193 (local address 127.0.0.1:56674)
0808 08:27:40.554067 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:40.554113 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:40.554245 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:40.554709 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:27:40.554715 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:27:40.555132 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:27:40.555257 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:40.555264 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:40.556517 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:40.556521 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:40.557510 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:40.557522 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:40.558188 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:40.558192 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:40.558297 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:40.558786 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:27:40.558802 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:27:40.560248 (+ 1446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:27:40.562044 (+ 1796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:40.562052 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:40.562058 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:40.562352 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:40.562630 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:40.562633 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:40.562635 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:40.562716 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:40.563151 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:27:40.563157 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:27:40.563269 (+ 112us) client_negotiation.cc:769] Sending connection context
0808 08:27:40.563382 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 08:27:40.563572 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":107,"threads_started":1}
I20260808 08:27:40.563771 15678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:40.553593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56674 (local address 127.22.235.193:44193)
0808 08:27:40.554183 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:40.554195 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:40.554284 (+ 89us) server_negotiation.cc:407] Connection header received
0808 08:27:40.554354 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:40.554359 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:40.554562 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:40.554697 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:27:40.555374 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:40.556395 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:40.557642 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:40.558079 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:40.558160 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:40.560373 (+ 2213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:27:40.560382 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:27:40.560391 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:27:40.560411 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:27:40.561947 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:40.562460 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:40.562466 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:40.562467 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:40.562549 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:40.562793 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:40.562797 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:40.562799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:40.563045 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:27:40.563158 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:40.563474 (+ 316us) server_negotiation.cc:299] Negotiation successful
0808 08:27:40.563636 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":100,"threads_started":1}
I20260808 08:27:40.564759 15559 tablet_service.cc:1514] Processing DeleteTablet for tablet 5378eaa5aaa34fd8a04e01cd8a3d8c1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:27:40 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:56674
I20260808 08:27:40.565719 15680 tablet_replica.cc:330] stopping tablet replica
I20260808 08:27:40.565997 15680 raft_consensus.cc:2229] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P b2e3f9bc786849869ce8e2432df29b53 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 08:27:40.566218 15680 raft_consensus.cc:2258] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P b2e3f9bc786849869ce8e2432df29b53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:27:40.567147 15680 ts_tablet_manager.cc:1905] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P b2e3f9bc786849869ce8e2432df29b53: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:27:40.570711 15680 ts_tablet_manager.cc:1918] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P b2e3f9bc786849869ce8e2432df29b53: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 08:27:40.570811 15680 log.cc:1196] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P b2e3f9bc786849869ce8e2432df29b53: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-0/wal/wals/5378eaa5aaa34fd8a04e01cd8a3d8c1a
I20260808 08:27:40.571080 15680 ts_tablet_manager.cc:1939] T 5378eaa5aaa34fd8a04e01cd8a3d8c1a P b2e3f9bc786849869ce8e2432df29b53: Deleting consensus metadata
I20260808 08:27:40.572745 15337 catalog_manager.cc:4841] TS b2e3f9bc786849869ce8e2432df29b53 (127.22.235.193:44193): tablet 5378eaa5aaa34fd8a04e01cd8a3d8c1a (table test-table [id=ee9efc8b062f43038e0a6998eecce16d]) successfully deleted
W20260808 08:27:40.652159 15674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:27:40.652423 15674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:27:40.652465 15674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:27:40.656390 15674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:27:40.656441 15674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:27:40.656500 15674 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.194
I20260808 08:27:40.661592 15674 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:40861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab
--principal=kudu/127.22.235.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.194:42205
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.235.194
--webserver_port=42477
--tserver_master_addrs=127.22.235.254:46193
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15674
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.194
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:27:40.662634 15674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:27:40.663637 15674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:27:40.673018 15695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:27:40.673074 15674 server_base.cc:1034] running on GCE node
W20260808 08:27:40.673079 15692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:27:40.673023 15691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:27:40.673605 15674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:27:40.674182 15674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:27:40.675331 15674 hybrid_clock.cc:648] HybridClock initialized: now 1786177660675305 us; error 41 us; skew 500 ppm
Aug 08 08:27:40 dist-test-slave-f7th krb5kdc[14368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177660, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:27:40.677947 15674 init.cc:377] Logged in from keytab as kudu/127.22.235.194@KRBTEST.COM (short username kudu)
I20260808 08:27:40.678810 15674 webserver.cc:460] Webserver started at http://127.22.235.194:42477/ using document root <none> and password file <none>
I20260808 08:27:40.679347 15674 fs_manager.cc:359] Metadata directory not provided
I20260808 08:27:40.679432 15674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:27:40.683241 15674 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:27:40.685220 15707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:40.686378 15674 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:27:40.686499 15674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-1/wal
uuid: "a6d6813abd514f13a3e5880174713c3a"
format_stamp: "Formatted at 2026-08-08 08:27:39 on dist-test-slave-f7th"
server_key: "8305ba8ff1f3802b214318a90262e41f"
server_key_iv: "8dc3c34822d4a9cf8efbcb79d34413c6"
server_key_version: "encryptionkey@0"
I20260808 08:27:40.687021 15674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:27:40.714515 15674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:27:40.717744 15674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:27:40.717932 15674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:27:40.718537 15674 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:27:40.719379 15674 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:27:40.719429 15674 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:40.719494 15674 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:27:40.719522 15674 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:40.728045 15674 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.194:42205
I20260808 08:27:40.728083 15882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.194:42205 every 8 connection(s)
I20260808 08:27:40.729049 15674 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
I20260808 08:27:40.732070 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 15674
I20260808 08:27:40.732451 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.195:37593
--local_ip_for_outbound_sockets=127.22.235.195
--tserver_master_addrs=127.22.235.254:46193
--webserver_port=43719
--webserver_interface=127.22.235.195
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab
--principal=kudu/127.22.235.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:40861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 08:27:40 dist-test-slave-f7th krb5kdc[14368](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177660, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.194@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:27:40.743497 15672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:40.730846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:54607 (local address 127.22.235.254:46193)
0808 08:27:40.730991 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:40.730996 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:40.731818 (+ 822us) server_negotiation.cc:407] Connection header received
0808 08:27:40.733255 (+ 1437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:40.733259 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:40.733321 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:40.733397 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:27:40.734831 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:40.735393 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:40.736579 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:40.736991 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:40.737106 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:40.739923 (+ 2817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:27:40.739927 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:27:40.739930 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:27:40.739965 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:27:40.741507 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:40.741973 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:40.741977 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:40.741979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:40.742075 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:40.742363 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:40.742366 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:40.742368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:40.742598 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:27:40.742705 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:40.743144 (+ 439us) server_negotiation.cc:299] Negotiation successful
0808 08:27:40.743285 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 08:27:40.744207 15886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:40.731208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.194:54607)
0808 08:27:40.731668 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:40.731708 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:40.733043 (+ 1335us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:40.733602 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:27:40.733609 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:27:40.733969 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:27:40.734650 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:40.734667 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:40.735544 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:40.735548 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:40.736440 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:40.736451 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:40.737147 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:40.737152 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:40.737346 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:40.738119 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:27:40.738140 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:27:40.739738 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:27:40.741616 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:40.741621 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:40.741633 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:40.741881 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:40.742158 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:40.742160 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:40.742161 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:40.742282 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:40.742686 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:27:40.742691 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:27:40.742918 (+ 227us) client_negotiation.cc:769] Sending connection context
0808 08:27:40.743067 (+ 149us) client_negotiation.cc:240] Negotiation successful
0808 08:27:40.743315 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":118,"threads_started":1}
I20260808 08:27:40.745422 15883 heartbeater.cc:347] Connected to a master server at 127.22.235.254:46193
I20260808 08:27:40.745683 15883 heartbeater.cc:464] Registering TS with master...
I20260808 08:27:40.746238 15883 heartbeater.cc:511] Master 127.22.235.254:46193 requested a full tablet report, sending...
I20260808 08:27:40.747367 15367 ts_manager.cc:194] Registered new tserver with Master: a6d6813abd514f13a3e5880174713c3a (127.22.235.194:42205)
I20260808 08:27:40.748099 15367 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.194@KRBTEST.COM'} at 127.22.235.194:54607
W20260808 08:27:40.849885 15887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:27:40.850134 15887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:27:40.850174 15887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:27:40.854122 15887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:27:40.854177 15887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:27:40.854233 15887 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.195
I20260808 08:27:40.859508 15887 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:40861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab
--principal=kudu/127.22.235.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.195:37593
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.235.195
--webserver_port=43719
--tserver_master_addrs=127.22.235.254:46193
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.195
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:27:40.860509 15887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:27:40.861598 15887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:27:40.871496 15902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:27:40.871466 15900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:27:40.871479 15897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:27:40.871824 15887 server_base.cc:1034] running on GCE node
I20260808 08:27:40.872212 15887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:27:40.872864 15887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:27:40.874044 15887 hybrid_clock.cc:648] HybridClock initialized: now 1786177660874025 us; error 29 us; skew 500 ppm
Aug 08 08:27:40 dist-test-slave-f7th krb5kdc[14368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177660, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:27:40.876842 15887 init.cc:377] Logged in from keytab as kudu/127.22.235.195@KRBTEST.COM (short username kudu)
I20260808 08:27:40.877696 15887 webserver.cc:460] Webserver started at http://127.22.235.195:43719/ using document root <none> and password file <none>
I20260808 08:27:40.878237 15887 fs_manager.cc:359] Metadata directory not provided
I20260808 08:27:40.878321 15887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:27:40.882302 15887 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 08:27:40.884656 15912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:40.885879 15887 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 08:27:40.885989 15887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-2/wal
uuid: "a8037a31939547ffa83bb358dc39307e"
format_stamp: "Formatted at 2026-08-08 08:27:39 on dist-test-slave-f7th"
server_key: "35da105d6b2ff23bd8af75eca5fed51a"
server_key_iv: "2ec23ad4c3b20e937be4c244ffdfe840"
server_key_version: "encryptionkey@0"
I20260808 08:27:40.886502 15887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:27:40.900604 15887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:27:40.903646 15887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:27:40.903823 15887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:27:40.904397 15887 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:27:40.905287 15887 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:27:40.905337 15887 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:40.905392 15887 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:27:40.905419 15887 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:40.913836 15887 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.195:37593
I20260808 08:27:40.913882 16098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.195:37593 every 8 connection(s)
I20260808 08:27:40.914814 15887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
I20260808 08:27:40.919824 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 15887
Aug 08 08:27:40 dist-test-slave-f7th krb5kdc[14368](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177660, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.195@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:27:40.928103 16109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:40.920494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55122 (local address 127.22.235.254:46193)
0808 08:27:40.920821 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:40.920825 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:40.920837 (+ 12us) server_negotiation.cc:407] Connection header received
0808 08:27:40.920982 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:40.920985 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:40.921029 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:40.921101 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:27:40.921808 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:40.922306 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:40.923127 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:40.923338 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:40.923420 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:40.924761 (+ 1341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:27:40.924765 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:27:40.924768 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:27:40.924792 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:27:40.926410 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:40.926811 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:40.926814 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:40.926816 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:40.926904 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:40.927199 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:40.927202 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:40.927204 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:40.927412 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:27:40.927574 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:40.927769 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 08:27:40.927889 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":74,"threads_started":1}
I20260808 08:27:40.928579 15672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:40.916552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:40473 (local address 127.22.235.254:46193)
0808 08:27:40.916725 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:40.916730 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:40.917568 (+ 838us) server_negotiation.cc:407] Connection header received
0808 08:27:40.918845 (+ 1277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:40.918848 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:40.918896 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:40.918967 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:27:40.920297 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:40.920974 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:40.922084 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:40.922310 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:40.922396 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:40.924945 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:27:40.924948 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:27:40.924950 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:27:40.924985 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:27:40.926429 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:40.927007 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:40.927011 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:40.927013 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:40.927106 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:40.927459 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:40.927462 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:40.927464 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:40.927658 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:27:40.927747 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:40.928336 (+ 589us) server_negotiation.cc:299] Negotiation successful
0808 08:27:40.928441 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 08:27:40.929522 16105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:40.916954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.195:40473)
0808 08:27:40.917418 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:40.917466 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:40.918661 (+ 1195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:40.919117 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:27:40.919124 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:27:40.919463 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:27:40.920136 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:40.920157 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:40.921100 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:40.921103 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:40.921958 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:40.921968 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:40.922447 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:40.922451 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:40.922607 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:40.923137 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:27:40.923153 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:27:40.924788 (+ 1635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:27:40.926536 (+ 1748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:40.926542 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:40.926557 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:40.926832 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:40.927201 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:40.927204 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:40.927206 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:40.927366 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:40.927755 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:27:40.927762 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:27:40.928059 (+ 297us) client_negotiation.cc:769] Sending connection context
0808 08:27:40.928296 (+ 237us) client_negotiation.cc:240] Negotiation successful
0808 08:27:40.928655 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":111,"threads_started":1}
I20260808 08:27:40.930637 16100 heartbeater.cc:347] Connected to a master server at 127.22.235.254:46193
I20260808 08:27:40.930850 16100 heartbeater.cc:464] Registering TS with master...
I20260808 08:27:40.931294 16100 heartbeater.cc:511] Master 127.22.235.254:46193 requested a full tablet report, sending...
I20260808 08:27:40.932235 15367 ts_manager.cc:194] Registered new tserver with Master: a8037a31939547ffa83bb358dc39307e (127.22.235.195:37593)
I20260808 08:27:40.932780 15367 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.195@KRBTEST.COM'} at 127.22.235.195:40473
I20260808 08:27:40.941581 23471 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 08:27:40.945531 15672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:40.942173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55136 (local address 127.22.235.254:46193)
0808 08:27:40.942284 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:40.942288 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:40.942454 (+ 166us) server_negotiation.cc:407] Connection header received
0808 08:27:40.942589 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:40.942591 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:40.942688 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:40.942772 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:27:40.942963 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:40.943443 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:40.944270 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:40.944467 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:40.944540 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:40.944833 (+ 293us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:27:40.945232 (+ 399us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 08:27:40.945322 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:40.945377 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 08:27:40.945422 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 08:27:40.946133 15367 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:55136:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 08:27:40.949371 15367 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 08:27:40.957340 15559 tablet_service.cc:1467] Processing CreateTablet for tablet ed717835f2824ad584c6319d2636a81c (DEFAULT_TABLE table=test-table [id=a95b8af1e66e44968f9788c337137785]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:27:40.958069 15559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed717835f2824ad584c6319d2636a81c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:27:40.960980 15484 tablet_bootstrap.cc:492] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53: Bootstrap starting.
I20260808 08:27:40.961840 16117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:40.956420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.194:42205 (local address 127.0.0.1:51350)
0808 08:27:40.956725 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:40.956738 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:40.956846 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:40.957436 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:27:40.957439 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:27:40.957460 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:27:40.957560 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:40.957567 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:40.959104 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:40.959108 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:40.960311 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:40.960321 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:40.961386 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:40.961390 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:40.961480 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:40.961528 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 08:27:40.961612 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 08:27:40.961685 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":117,"threads_started":1}
I20260808 08:27:40.961923 15484 tablet_bootstrap.cc:654] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53: Neither blocks nor log segments found. Creating new log.
I20260808 08:27:40.961916 16118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:40.956596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51350 (local address 127.22.235.194:42205)
0808 08:27:40.956997 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:40.957010 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:40.957036 (+ 26us) server_negotiation.cc:407] Connection header received
0808 08:27:40.957110 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:40.957115 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:40.957283 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:40.957411 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:27:40.957880 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:40.958944 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:40.960429 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:40.961174 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:40.961298 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:40.961488 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:40.961663 (+ 175us) server_negotiation.cc:299] Negotiation successful
0808 08:27:40.961769 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":170,"threads_started":1}
I20260808 08:27:40.962684 15677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:40.956163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.195:37593 (local address 127.0.0.1:39076)
0808 08:27:40.956301 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:40.956317 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:40.956397 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:40.958222 (+ 1825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:27:40.958225 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:27:40.958246 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:27:40.958342 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:40.958347 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:40.959631 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:40.959634 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:40.960760 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:40.960772 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:40.962292 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:40.962296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:40.962398 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:40.962424 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 08:27:40.962492 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 08:27:40.962571 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 08:27:40.963445 16119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:40.956759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39076 (local address 127.22.235.195:37593)
0808 08:27:40.957769 (+ 1010us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:40.957781 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:40.957806 (+ 25us) server_negotiation.cc:407] Connection header received
0808 08:27:40.957886 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:40.957892 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:40.958072 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:40.958482 (+ 410us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:27:40.958612 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:40.959458 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:40.960912 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:40.961838 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:40.963036 (+ 1198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:40.963169 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:40.963223 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 08:27:40.963311 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":904,"thread_start_us":105,"threads_started":1}
I20260808 08:27:40.963558 15484 tablet_bootstrap.cc:492] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53: No bootstrap required, opened a new log
I20260808 08:27:40.963653 15484 ts_tablet_manager.cc:1397] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 08:27:40.964049 15484 raft_consensus.cc:348] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8037a31939547ffa83bb358dc39307e" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37593 } } peers { permanent_uuid: "a6d6813abd514f13a3e5880174713c3a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 42205 } } peers { permanent_uuid: "b2e3f9bc786849869ce8e2432df29b53" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44193 } }
I20260808 08:27:40.964149 15484 raft_consensus.cc:374] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:27:40.964186 15484 raft_consensus.cc:729] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e3f9bc786849869ce8e2432df29b53, State: Initialized, Role: FOLLOWER
I20260808 08:27:40.964321 15484 consensus_queue.cc:260] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8037a31939547ffa83bb358dc39307e" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37593 } } peers { permanent_uuid: "a6d6813abd514f13a3e5880174713c3a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 42205 } } peers { permanent_uuid: "b2e3f9bc786849869ce8e2432df29b53" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44193 } }
I20260808 08:27:40.964569 15484 ts_tablet_manager.cc:1428] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 08:27:40.965142 15791 tablet_service.cc:1467] Processing CreateTablet for tablet ed717835f2824ad584c6319d2636a81c (DEFAULT_TABLE table=test-table [id=a95b8af1e66e44968f9788c337137785]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:27:40.966338 15791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed717835f2824ad584c6319d2636a81c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:27:40.967249 15992 tablet_service.cc:1467] Processing CreateTablet for tablet ed717835f2824ad584c6319d2636a81c (DEFAULT_TABLE table=test-table [id=a95b8af1e66e44968f9788c337137785]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:27:40.968075 15992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed717835f2824ad584c6319d2636a81c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:27:40.973214 16121 tablet_bootstrap.cc:492] T ed717835f2824ad584c6319d2636a81c P a6d6813abd514f13a3e5880174713c3a: Bootstrap starting.
I20260808 08:27:40.974332 16122 tablet_bootstrap.cc:492] T ed717835f2824ad584c6319d2636a81c P a8037a31939547ffa83bb358dc39307e: Bootstrap starting.
I20260808 08:27:40.976126 16121 tablet_bootstrap.cc:654] T ed717835f2824ad584c6319d2636a81c P a6d6813abd514f13a3e5880174713c3a: Neither blocks nor log segments found. Creating new log.
I20260808 08:27:40.976953 16122 tablet_bootstrap.cc:654] T ed717835f2824ad584c6319d2636a81c P a8037a31939547ffa83bb358dc39307e: Neither blocks nor log segments found. Creating new log.
I20260808 08:27:40.977089 16121 log.cc:824] T ed717835f2824ad584c6319d2636a81c P a6d6813abd514f13a3e5880174713c3a: Log is configured to *not* fsync() on all Append() calls
I20260808 08:27:40.977739 16122 log.cc:824] T ed717835f2824ad584c6319d2636a81c P a8037a31939547ffa83bb358dc39307e: Log is configured to *not* fsync() on all Append() calls
I20260808 08:27:40.979080 16121 tablet_bootstrap.cc:492] T ed717835f2824ad584c6319d2636a81c P a6d6813abd514f13a3e5880174713c3a: No bootstrap required, opened a new log
I20260808 08:27:40.979297 16121 ts_tablet_manager.cc:1397] T ed717835f2824ad584c6319d2636a81c P a6d6813abd514f13a3e5880174713c3a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 08:27:40.979643 16122 tablet_bootstrap.cc:492] T ed717835f2824ad584c6319d2636a81c P a8037a31939547ffa83bb358dc39307e: No bootstrap required, opened a new log
I20260808 08:27:40.979842 16122 ts_tablet_manager.cc:1397] T ed717835f2824ad584c6319d2636a81c P a8037a31939547ffa83bb358dc39307e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 08:27:40.983246 16121 raft_consensus.cc:348] T ed717835f2824ad584c6319d2636a81c P a6d6813abd514f13a3e5880174713c3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8037a31939547ffa83bb358dc39307e" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37593 } } peers { permanent_uuid: "a6d6813abd514f13a3e5880174713c3a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 42205 } } peers { permanent_uuid: "b2e3f9bc786849869ce8e2432df29b53" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44193 } }
I20260808 08:27:40.983537 16121 raft_consensus.cc:374] T ed717835f2824ad584c6319d2636a81c P a6d6813abd514f13a3e5880174713c3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:27:40.983603 16121 raft_consensus.cc:729] T ed717835f2824ad584c6319d2636a81c P a6d6813abd514f13a3e5880174713c3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6d6813abd514f13a3e5880174713c3a, State: Initialized, Role: FOLLOWER
I20260808 08:27:40.983501 16122 raft_consensus.cc:348] T ed717835f2824ad584c6319d2636a81c P a8037a31939547ffa83bb358dc39307e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8037a31939547ffa83bb358dc39307e" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37593 } } peers { permanent_uuid: "a6d6813abd514f13a3e5880174713c3a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 42205 } } peers { permanent_uuid: "b2e3f9bc786849869ce8e2432df29b53" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44193 } }
I20260808 08:27:40.983732 16122 raft_consensus.cc:374] T ed717835f2824ad584c6319d2636a81c P a8037a31939547ffa83bb358dc39307e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:27:40.983803 16122 raft_consensus.cc:729] T ed717835f2824ad584c6319d2636a81c P a8037a31939547ffa83bb358dc39307e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8037a31939547ffa83bb358dc39307e, State: Initialized, Role: FOLLOWER
I20260808 08:27:40.984079 16121 consensus_queue.cc:260] T ed717835f2824ad584c6319d2636a81c P a6d6813abd514f13a3e5880174713c3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8037a31939547ffa83bb358dc39307e" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37593 } } peers { permanent_uuid: "a6d6813abd514f13a3e5880174713c3a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 42205 } } peers { permanent_uuid: "b2e3f9bc786849869ce8e2432df29b53" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44193 } }
I20260808 08:27:40.984169 16122 consensus_queue.cc:260] T ed717835f2824ad584c6319d2636a81c P a8037a31939547ffa83bb358dc39307e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8037a31939547ffa83bb358dc39307e" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37593 } } peers { permanent_uuid: "a6d6813abd514f13a3e5880174713c3a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 42205 } } peers { permanent_uuid: "b2e3f9bc786849869ce8e2432df29b53" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44193 } }
I20260808 08:27:40.984812 16100 heartbeater.cc:503] Master 127.22.235.254:46193 was elected leader, sending a full tablet report...
I20260808 08:27:40.984925 15883 heartbeater.cc:503] Master 127.22.235.254:46193 was elected leader, sending a full tablet report...
I20260808 08:27:40.985162 16122 ts_tablet_manager.cc:1428] T ed717835f2824ad584c6319d2636a81c P a8037a31939547ffa83bb358dc39307e: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 08:27:40.986088 16121 ts_tablet_manager.cc:1428] T ed717835f2824ad584c6319d2636a81c P a6d6813abd514f13a3e5880174713c3a: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
W20260808 08:27:41.027956 15670 tablet.cc:2369] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 08:27:41.118775 16132 raft_consensus.cc:482] T ed717835f2824ad584c6319d2636a81c P a8037a31939547ffa83bb358dc39307e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 08:27:41.118996 16132 raft_consensus.cc:504] T ed717835f2824ad584c6319d2636a81c P a8037a31939547ffa83bb358dc39307e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8037a31939547ffa83bb358dc39307e" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37593 } } peers { permanent_uuid: "a6d6813abd514f13a3e5880174713c3a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 42205 } } peers { permanent_uuid: "b2e3f9bc786849869ce8e2432df29b53" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44193 } }
I20260808 08:27:41.120128 16132 leader_election.cc:288] T ed717835f2824ad584c6319d2636a81c P a8037a31939547ffa83bb358dc39307e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6d6813abd514f13a3e5880174713c3a (127.22.235.194:42205), b2e3f9bc786849869ce8e2432df29b53 (127.22.235.193:44193)
I20260808 08:27:41.124161 16105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:41.120377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.194:42205 (local address 127.22.235.195:51211)
0808 08:27:41.120511 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:41.120526 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:41.120642 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:41.120929 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:27:41.120932 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:27:41.120954 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:27:41.121075 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:41.121081 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:41.121999 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:41.122002 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:41.122944 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:41.122954 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:41.123736 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:41.123739 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:41.123854 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:41.123882 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 08:27:41.123958 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 08:27:41.124030 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 08:27:41.124161 16118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:41.120471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:51211 (local address 127.22.235.194:42205)
0808 08:27:41.120600 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:41.120603 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:41.120636 (+ 33us) server_negotiation.cc:407] Connection header received
0808 08:27:41.120749 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:41.120752 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:41.120807 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:41.120898 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:27:41.121203 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:41.121883 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:41.123078 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:41.123626 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:41.123714 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:41.123758 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:41.123989 (+ 231us) server_negotiation.cc:299] Negotiation successful
0808 08:27:41.124053 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 08:27:41.124727 16133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:41.120575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:48959 (local address 127.22.235.193:44193)
0808 08:27:41.120929 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:41.120934 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:41.121196 (+ 262us) server_negotiation.cc:407] Connection header received
0808 08:27:41.121318 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:41.121320 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:41.121375 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:41.121459 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:27:41.121721 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:41.122368 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:41.123513 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:41.124099 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:41.124216 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:41.124377 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:41.124508 (+ 131us) server_negotiation.cc:299] Negotiation successful
0808 08:27:41.124576 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":166,"threads_started":1}
I20260808 08:27:41.124938 15820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed717835f2824ad584c6319d2636a81c" candidate_uuid: "a8037a31939547ffa83bb358dc39307e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6d6813abd514f13a3e5880174713c3a" is_pre_election: true
I20260808 08:27:41.125082 16134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:41.120392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:44193 (local address 127.22.235.195:48959)
0808 08:27:41.121115 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:41.121127 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:41.121226 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:41.121487 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:27:41.121490 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:27:41.121508 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:27:41.121628 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:41.121633 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:41.122491 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:41.122494 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:41.123390 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:41.123400 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:41.124249 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:41.124253 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:41.124350 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:41.124374 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 08:27:41.124866 (+ 492us) client_negotiation.cc:240] Negotiation successful
0808 08:27:41.124942 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"mutex_wait_us":332,"thread_start_us":113,"threads_started":1}
I20260808 08:27:41.125275 15820 raft_consensus.cc:2454] T ed717835f2824ad584c6319d2636a81c P a6d6813abd514f13a3e5880174713c3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8037a31939547ffa83bb358dc39307e in term 0.
I20260808 08:27:41.125597 15595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed717835f2824ad584c6319d2636a81c" candidate_uuid: "a8037a31939547ffa83bb358dc39307e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e3f9bc786849869ce8e2432df29b53" is_pre_election: true
I20260808 08:27:41.125777 15921 leader_election.cc:302] T ed717835f2824ad584c6319d2636a81c P a8037a31939547ffa83bb358dc39307e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6d6813abd514f13a3e5880174713c3a, a8037a31939547ffa83bb358dc39307e; no voters:
I20260808 08:27:41.125869 15595 raft_consensus.cc:2454] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a8037a31939547ffa83bb358dc39307e in term 0.
I20260808 08:27:41.126034 16132 raft_consensus.cc:2781] T ed717835f2824ad584c6319d2636a81c P a8037a31939547ffa83bb358dc39307e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 08:27:41.126124 16132 raft_consensus.cc:482] T ed717835f2824ad584c6319d2636a81c P a8037a31939547ffa83bb358dc39307e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 08:27:41.126183 16132 raft_consensus.cc:3037] T ed717835f2824ad584c6319d2636a81c P a8037a31939547ffa83bb358dc39307e [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:27:41.127269 16132 raft_consensus.cc:504] T ed717835f2824ad584c6319d2636a81c P a8037a31939547ffa83bb358dc39307e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8037a31939547ffa83bb358dc39307e" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37593 } } peers { permanent_uuid: "a6d6813abd514f13a3e5880174713c3a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 42205 } } peers { permanent_uuid: "b2e3f9bc786849869ce8e2432df29b53" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44193 } }
I20260808 08:27:41.127590 16132 leader_election.cc:288] T ed717835f2824ad584c6319d2636a81c P a8037a31939547ffa83bb358dc39307e [CANDIDATE]: Term 1 election: Requested vote from peers a6d6813abd514f13a3e5880174713c3a (127.22.235.194:42205), b2e3f9bc786849869ce8e2432df29b53 (127.22.235.193:44193)
I20260808 08:27:41.127928 15820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed717835f2824ad584c6319d2636a81c" candidate_uuid: "a8037a31939547ffa83bb358dc39307e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6d6813abd514f13a3e5880174713c3a"
I20260808 08:27:41.127951 15595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed717835f2824ad584c6319d2636a81c" candidate_uuid: "a8037a31939547ffa83bb358dc39307e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e3f9bc786849869ce8e2432df29b53"
I20260808 08:27:41.128068 15595 raft_consensus.cc:3037] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:27:41.128067 15820 raft_consensus.cc:3037] T ed717835f2824ad584c6319d2636a81c P a6d6813abd514f13a3e5880174713c3a [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:27:41.129326 15595 raft_consensus.cc:2454] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8037a31939547ffa83bb358dc39307e in term 1.
I20260808 08:27:41.129326 15820 raft_consensus.cc:2454] T ed717835f2824ad584c6319d2636a81c P a6d6813abd514f13a3e5880174713c3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8037a31939547ffa83bb358dc39307e in term 1.
I20260808 08:27:41.129634 15919 leader_election.cc:302] T ed717835f2824ad584c6319d2636a81c P a8037a31939547ffa83bb358dc39307e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8037a31939547ffa83bb358dc39307e, b2e3f9bc786849869ce8e2432df29b53; no voters:
I20260808 08:27:41.129860 16132 raft_consensus.cc:2781] T ed717835f2824ad584c6319d2636a81c P a8037a31939547ffa83bb358dc39307e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:27:41.130074 16132 raft_consensus.cc:686] T ed717835f2824ad584c6319d2636a81c P a8037a31939547ffa83bb358dc39307e [term 1 LEADER]: Becoming Leader. State: Replica: a8037a31939547ffa83bb358dc39307e, State: Running, Role: LEADER
I20260808 08:27:41.130378 16132 consensus_queue.cc:237] T ed717835f2824ad584c6319d2636a81c P a8037a31939547ffa83bb358dc39307e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8037a31939547ffa83bb358dc39307e" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37593 } } peers { permanent_uuid: "a6d6813abd514f13a3e5880174713c3a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 42205 } } peers { permanent_uuid: "b2e3f9bc786849869ce8e2432df29b53" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44193 } }
I20260808 08:27:41.133574 15363 catalog_manager.cc:5495] T ed717835f2824ad584c6319d2636a81c P a8037a31939547ffa83bb358dc39307e reported cstate change: term changed from 0 to 1, leader changed from <none> to a8037a31939547ffa83bb358dc39307e (127.22.235.195). New cstate: current_term: 1 leader_uuid: "a8037a31939547ffa83bb358dc39307e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8037a31939547ffa83bb358dc39307e" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6d6813abd514f13a3e5880174713c3a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 42205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2e3f9bc786849869ce8e2432df29b53" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44193 } health_report { overall_health: UNKNOWN } } }
I20260808 08:27:41.151793 16119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:41.148811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39078 (local address 127.22.235.195:37593)
0808 08:27:41.148936 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:41.148939 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:41.148953 (+ 14us) server_negotiation.cc:407] Connection header received
0808 08:27:41.149065 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:41.149067 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:41.149119 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:41.149202 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:27:41.149371 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:41.149847 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:41.150604 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:41.150801 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:41.150869 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:41.151102 (+ 233us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:27:41.151498 (+ 396us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 08:27:41.151583 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:41.151613 (+ 30us) server_negotiation.cc:299] Negotiation successful
0808 08:27:41.151660 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 08:27:41.157436 15595 raft_consensus.cc:1261] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53 [term 1 FOLLOWER]: Refusing update from remote peer a8037a31939547ffa83bb358dc39307e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:27:41.157441 15820 raft_consensus.cc:1261] T ed717835f2824ad584c6319d2636a81c P a6d6813abd514f13a3e5880174713c3a [term 1 FOLLOWER]: Refusing update from remote peer a8037a31939547ffa83bb358dc39307e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:27:41.158105 16132 consensus_queue.cc:1035] T ed717835f2824ad584c6319d2636a81c P a8037a31939547ffa83bb358dc39307e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e3f9bc786849869ce8e2432df29b53" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:27:41.158285 16136 consensus_queue.cc:1035] T ed717835f2824ad584c6319d2636a81c P a8037a31939547ffa83bb358dc39307e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6d6813abd514f13a3e5880174713c3a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 42205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:27:41.163188 16139 mvcc.cc:205] Tried to move back new op lower bound from 7316183700093915136 to 7316183699992649728. Current Snapshot: MvccSnapshot[applied={T|T < 7316183700093915136}]
I20260808 08:27:41.175719 15367 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:55136:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:27:41.175894 15367 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:55136:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:27:41.178200 15367 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1: Sending DeleteTablet for 3 replicas of tablet ed717835f2824ad584c6319d2636a81c
I20260808 08:27:41.178866 15992 tablet_service.cc:1514] Processing DeleteTablet for tablet ed717835f2824ad584c6319d2636a81c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:27:41 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:39076
I20260808 08:27:41.178965 15559 tablet_service.cc:1514] Processing DeleteTablet for tablet ed717835f2824ad584c6319d2636a81c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:27:41 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:56674
I20260808 08:27:41.179363 16160 tablet_replica.cc:330] T ed717835f2824ad584c6319d2636a81c P a8037a31939547ffa83bb358dc39307e: stopping tablet replica
I20260808 08:27:41.179384 15791 tablet_service.cc:1514] Processing DeleteTablet for tablet ed717835f2824ad584c6319d2636a81c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:27:41 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:51350
I20260808 08:27:41.179677 16160 raft_consensus.cc:2229] T ed717835f2824ad584c6319d2636a81c P a8037a31939547ffa83bb358dc39307e [term 1 LEADER]: Raft consensus shutting down.
I20260808 08:27:41.179869 16163 tablet_replica.cc:330] T ed717835f2824ad584c6319d2636a81c P a6d6813abd514f13a3e5880174713c3a: stopping tablet replica
I20260808 08:27:41.180008 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 15425
I20260808 08:27:41.180030 16160 raft_consensus.cc:2258] T ed717835f2824ad584c6319d2636a81c P a8037a31939547ffa83bb358dc39307e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:27:41.225710 16163 raft_consensus.cc:2229] T ed717835f2824ad584c6319d2636a81c P a6d6813abd514f13a3e5880174713c3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 08:27:41.226419 16163 raft_consensus.cc:2258] T ed717835f2824ad584c6319d2636a81c P a6d6813abd514f13a3e5880174713c3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:27:41.228229 16163 ts_tablet_manager.cc:1905] T ed717835f2824ad584c6319d2636a81c P a6d6813abd514f13a3e5880174713c3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:27:41.231189 16163 ts_tablet_manager.cc:1918] T ed717835f2824ad584c6319d2636a81c P a6d6813abd514f13a3e5880174713c3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 08:27:41.231374 16163 log.cc:1196] T ed717835f2824ad584c6319d2636a81c P a6d6813abd514f13a3e5880174713c3a: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-1/wal/wals/ed717835f2824ad584c6319d2636a81c
I20260808 08:27:41.231357 16160 ts_tablet_manager.cc:1905] T ed717835f2824ad584c6319d2636a81c P a8037a31939547ffa83bb358dc39307e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:27:41.231828 16163 ts_tablet_manager.cc:1939] T ed717835f2824ad584c6319d2636a81c P a6d6813abd514f13a3e5880174713c3a: Deleting consensus metadata
I20260808 08:27:41.233259 15341 catalog_manager.cc:4841] TS a6d6813abd514f13a3e5880174713c3a (127.22.235.194:42205): tablet ed717835f2824ad584c6319d2636a81c (table test-table [id=a95b8af1e66e44968f9788c337137785]) successfully deleted
I20260808 08:27:41.234021 16160 ts_tablet_manager.cc:1918] T ed717835f2824ad584c6319d2636a81c P a8037a31939547ffa83bb358dc39307e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 08:27:41.234122 16160 log.cc:1196] T ed717835f2824ad584c6319d2636a81c P a8037a31939547ffa83bb358dc39307e: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-2/wal/wals/ed717835f2824ad584c6319d2636a81c
I20260808 08:27:41.234436 16160 ts_tablet_manager.cc:1939] T ed717835f2824ad584c6319d2636a81c P a8037a31939547ffa83bb358dc39307e: Deleting consensus metadata
W20260808 08:27:41.234616 15337 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.235.193:44193 (error 108)
I20260808 08:27:41.235530 15336 catalog_manager.cc:4841] TS a8037a31939547ffa83bb358dc39307e (127.22.235.195:37593): tablet ed717835f2824ad584c6319d2636a81c (table test-table [id=a95b8af1e66e44968f9788c337137785]) successfully deleted
I20260808 08:27:41.235785 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 15674
I20260808 08:27:41.235827 15677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:41.235499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:44193 (local address 127.0.0.1:35348)
0808 08:27:41.235653 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:41.235736 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.193:44193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260808 08:27:41.242187 15337 catalog_manager.cc:4584] TS b2e3f9bc786849869ce8e2432df29b53 (127.22.235.193:44193): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ed717835f2824ad584c6319d2636a81c: Network error: Client connection negotiation failed: client connection to 127.22.235.193:44193: connect: Connection refused (error 111)
I20260808 08:27:41.265340 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 15887
I20260808 08:27:41.279523 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 15305
I20260808 08:27:41.288671 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:46193
--webserver_interface=127.22.235.254
--webserver_port=33717
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:40861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:46193
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:27:41.398742 16165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:27:41.398979 16165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:27:41.399044 16165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:27:41.402932 16165 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:27:41.402985 16165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:27:41.403017 16165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:27:41.403040 16165 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:27:41.403069 16165 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:27:41.408185 16165 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:40861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:46193
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:46193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.235.254
--webserver_port=33717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16165
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:27:41.409200 16165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:27:41.410145 16165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:27:41.418304 16181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:27:41.418422 16165 server_base.cc:1034] running on GCE node
W20260808 08:27:41.418299 16179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:27:41.418295 16183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:27:41.419137 16165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:27:41.420203 16165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:27:41.421353 16165 hybrid_clock.cc:648] HybridClock initialized: now 1786177661421330 us; error 40 us; skew 500 ppm
Aug 08 08:27:41 dist-test-slave-f7th krb5kdc[14368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177661, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:27:41.423899 16165 init.cc:377] Logged in from keytab as kudu/127.22.235.254@KRBTEST.COM (short username kudu)
I20260808 08:27:41.424788 16165 webserver.cc:460] Webserver started at http://127.22.235.254:33717/ using document root <none> and password file <none>
I20260808 08:27:41.425320 16165 fs_manager.cc:359] Metadata directory not provided
I20260808 08:27:41.425387 16165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:27:41.429246 16165 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 08:27:41.431329 16196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:41.432523 16165 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:27:41.432673 16165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "44f372ca2df24ab08e3f6ffa192eb3e1"
format_stamp: "Formatted at 2026-08-08 08:27:38 on dist-test-slave-f7th"
server_key: "10dbbed96ad8730cc7fd8ed49f6c1c7d"
server_key_iv: "1682069a614bb82e2ee2706e29085580"
server_key_version: "encryptionkey@0"
I20260808 08:27:41.433190 16165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:27:41.462260 16165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:27:41.465265 16165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:27:41.465440 16165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:27:41.472141 16165 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.254:46193
I20260808 08:27:41.472376 16286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.254:46193 every 8 connection(s)
I20260808 08:27:41.473248 16165 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:27:41.474846 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 16165
I20260808 08:27:41.475298 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.193:44193
--local_ip_for_outbound_sockets=127.22.235.193
--tserver_master_addrs=127.22.235.254:46193
--webserver_port=42437
--webserver_interface=127.22.235.193
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab
--principal=kudu/127.22.235.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:40861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 08:27:41.478150 16288 sys_catalog.cc:263] Verifying existing consensus state
I20260808 08:27:41.479753 16288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1: Bootstrap starting.
I20260808 08:27:41.496809 16288 log.cc:824] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1: Log is configured to *not* fsync() on all Append() calls
I20260808 08:27:41.508037 16288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 08:27:41.508368 16288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1: Bootstrap complete.
I20260808 08:27:41.512240 16288 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44f372ca2df24ab08e3f6ffa192eb3e1" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46193 } }
I20260808 08:27:41.512639 16288 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44f372ca2df24ab08e3f6ffa192eb3e1, State: Initialized, Role: FOLLOWER
I20260808 08:27:41.513151 16288 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44f372ca2df24ab08e3f6ffa192eb3e1" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46193 } }
I20260808 08:27:41.513269 16288 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:27:41.513329 16288 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:27:41.513422 16288 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [term 2 FOLLOWER]: Advancing to term 3
I20260808 08:27:41.515161 16288 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44f372ca2df24ab08e3f6ffa192eb3e1" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46193 } }
I20260808 08:27:41.515458 16288 leader_election.cc:302] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44f372ca2df24ab08e3f6ffa192eb3e1; no voters:
I20260808 08:27:41.515741 16288 leader_election.cc:288] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 08:27:41.515837 16295 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 08:27:41.516052 16295 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [term 3 LEADER]: Becoming Leader. State: Replica: 44f372ca2df24ab08e3f6ffa192eb3e1, State: Running, Role: LEADER
I20260808 08:27:41.516336 16295 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44f372ca2df24ab08e3f6ffa192eb3e1" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46193 } }
I20260808 08:27:41.516978 16288 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:27:41.517326 16298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44f372ca2df24ab08e3f6ffa192eb3e1. Latest consensus state: current_term: 3 leader_uuid: "44f372ca2df24ab08e3f6ffa192eb3e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44f372ca2df24ab08e3f6ffa192eb3e1" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46193 } } }
I20260808 08:27:41.517448 16298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [sys.catalog]: This master's current role is: LEADER
I20260808 08:27:41.517690 16296 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "44f372ca2df24ab08e3f6ffa192eb3e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44f372ca2df24ab08e3f6ffa192eb3e1" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46193 } } }
I20260808 08:27:41.517787 16296 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44f372ca2df24ab08e3f6ffa192eb3e1 [sys.catalog]: This master's current role is: LEADER
I20260808 08:27:41.517836 16300 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:27:41.522922 16300 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:27:41.523571 16300 catalog_manager.cc:1259] Loaded cluster ID: bd2ab53661ea4a199c69e609adbf39ab
I20260808 08:27:41.523636 16300 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260808 08:27:41.524747 16300 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f6c102349ad google::LogMessage::Fail() at ??:0
@ 0x7f6c1023788a google::LogMessage::SendToLog() at ??:0
@ 0x7f6c102344de google::LogMessage::Flush() at ??:0
@ 0x7f6c10235a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f6c1ab77fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f6c1ab78d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f6c1ab75579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f6c1abb9640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f6c124863d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f6c12486c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f6c124883f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f6c12477512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f6c13abb6db start_thread at ??:0
@ 0x7f6c0ed3261f clone at ??:0
W20260808 08:27:41.601260 16291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:27:41.601518 16291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:27:41.601562 16291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:27:41.605696 16291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:27:41.605743 16291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:27:41.605798 16291 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.193
I20260808 08:27:41.611248 16291 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:40861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab
--principal=kudu/127.22.235.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.193:44193
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.235.193
--webserver_port=42437
--tserver_master_addrs=127.22.235.254:46193
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.193
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:27:41.612306 16291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:27:41.613438 16291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:27:41.623960 16349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:27:41.623980 16350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:27:41.624256 16291 server_base.cc:1034] running on GCE node
W20260808 08:27:41.624266 16353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:27:41.624691 16291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:27:41.625329 16291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:27:41.626510 16291 hybrid_clock.cc:648] HybridClock initialized: now 1786177661626490 us; error 33 us; skew 500 ppm
Aug 08 08:27:41 dist-test-slave-f7th krb5kdc[14368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177661, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:27:41.629268 16291 init.cc:377] Logged in from keytab as kudu/127.22.235.193@KRBTEST.COM (short username kudu)
I20260808 08:27:41.630148 16291 webserver.cc:460] Webserver started at http://127.22.235.193:42437/ using document root <none> and password file <none>
I20260808 08:27:41.630694 16291 fs_manager.cc:359] Metadata directory not provided
I20260808 08:27:41.630764 16291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:27:41.634671 16291 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260808 08:27:41.636802 16367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:41.638046 16291 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 08:27:41.638146 16291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-0/wal
uuid: "b2e3f9bc786849869ce8e2432df29b53"
format_stamp: "Formatted at 2026-08-08 08:27:39 on dist-test-slave-f7th"
server_key: "24ded7bead70c8e00ea7681813f84075"
server_key_iv: "7cfda73c74bfc09c77b0c8ec0b029c65"
server_key_version: "encryptionkey@0"
I20260808 08:27:41.638674 16291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:27:41.657259 16291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:27:41.660238 16291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:27:41.660410 16291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:27:41.661022 16291 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:27:41.662137 16379 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 08:27:41.667946 16291 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 08:27:41.668011 16291 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260808 08:27:41.668078 16291 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 08:27:41.669747 16291 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 08:27:41.669792 16291 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260808 08:27:41.670143 16379 tablet_bootstrap.cc:492] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53: Bootstrap starting.
I20260808 08:27:41.679594 16291 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.193:44193
I20260808 08:27:41.679873 16537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.193:44193 every 8 connection(s)
I20260808 08:27:41.680584 16291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
I20260808 08:27:41.682756 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 16291
I20260808 08:27:41.683209 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.194:42205
--local_ip_for_outbound_sockets=127.22.235.194
--tserver_master_addrs=127.22.235.254:46193
--webserver_port=42477
--webserver_interface=127.22.235.194
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab
--principal=kudu/127.22.235.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:40861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 08:27:41.691784 16379 log.cc:824] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53: Log is configured to *not* fsync() on all Append() calls
I20260808 08:27:41.701256 16379 tablet_bootstrap.cc:492] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 08:27:41.701694 16379 tablet_bootstrap.cc:492] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53: Bootstrap complete.
I20260808 08:27:41.703073 16379 ts_tablet_manager.cc:1397] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53: Time spent bootstrapping tablet: real 0.033s user 0.025s sys 0.005s
I20260808 08:27:41.706301 16379 raft_consensus.cc:348] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8037a31939547ffa83bb358dc39307e" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37593 } } peers { permanent_uuid: "a6d6813abd514f13a3e5880174713c3a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 42205 } } peers { permanent_uuid: "b2e3f9bc786849869ce8e2432df29b53" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44193 } }
I20260808 08:27:41.706851 16379 raft_consensus.cc:729] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e3f9bc786849869ce8e2432df29b53, State: Initialized, Role: FOLLOWER
I20260808 08:27:41.707405 16379 consensus_queue.cc:260] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8037a31939547ffa83bb358dc39307e" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37593 } } peers { permanent_uuid: "a6d6813abd514f13a3e5880174713c3a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 42205 } } peers { permanent_uuid: "b2e3f9bc786849869ce8e2432df29b53" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44193 } }
I20260808 08:27:41.708477 16379 ts_tablet_manager.cc:1428] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
W20260808 08:27:41.766067 16543 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:27:41.682489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.193:37063)
0808 08:27:41.683884 (+ 1395us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:41.683944 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:41.684939 (+ 995us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:41.765303 (+ 80364us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":1160,"thread_start_us":141,"threads_started":1}
W20260808 08:27:41.766597 16539 heartbeater.cc:644] Failed to heartbeat to 127.22.235.254:46193 (0 consecutive failures): Network error: Failed to ping master at 127.22.235.254:46193: Client connection negotiation failed: client connection to 127.22.235.254:46193: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 08:27:41.767409 16543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:41.767118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.193:54661)
0808 08:27:41.767280 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:41.767350 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260808 08:27:41.768065 16543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:41.767860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.193:48287)
0808 08:27:41.767961 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:41.768017 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 08:27:41.768213 16539 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 08:27:41.800411 16544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:27:41.800694 16544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:27:41.800745 16544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:27:41.804733 16544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:27:41.804786 16544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:27:41.804843 16544 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.194
I20260808 08:27:41.810082 16544 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:40861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab
--principal=kudu/127.22.235.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.194:42205
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.235.194
--webserver_port=42477
--tserver_master_addrs=127.22.235.254:46193
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16544
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.194
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:27:41.811115 16544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:27:41.812167 16544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:27:41.821851 16568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:27:41.821874 16561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:27:41.821940 16544 server_base.cc:1034] running on GCE node
W20260808 08:27:41.821856 16562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:27:41.822434 16544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:27:41.823124 16544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:27:41.824301 16544 hybrid_clock.cc:648] HybridClock initialized: now 1786177661824271 us; error 38 us; skew 500 ppm
Aug 08 08:27:41 dist-test-slave-f7th krb5kdc[14368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177661, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:27:41.827082 16544 init.cc:377] Logged in from keytab as kudu/127.22.235.194@KRBTEST.COM (short username kudu)
I20260808 08:27:41.827906 16544 webserver.cc:460] Webserver started at http://127.22.235.194:42477/ using document root <none> and password file <none>
I20260808 08:27:41.828433 16544 fs_manager.cc:359] Metadata directory not provided
I20260808 08:27:41.828501 16544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:27:41.832331 16544 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260808 08:27:41.834458 16582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:41.835676 16544 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 08:27:41.835784 16544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-1/wal
uuid: "a6d6813abd514f13a3e5880174713c3a"
format_stamp: "Formatted at 2026-08-08 08:27:39 on dist-test-slave-f7th"
server_key: "8305ba8ff1f3802b214318a90262e41f"
server_key_iv: "8dc3c34822d4a9cf8efbcb79d34413c6"
server_key_version: "encryptionkey@0"
I20260808 08:27:41.836309 16544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:27:41.861143 16544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:27:41.864431 16544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:27:41.864684 16544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:27:41.865337 16544 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:27:41.866211 16544 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:27:41.866261 16544 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:41.866317 16544 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:27:41.866348 16544 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:41.875025 16544 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.194:42205
I20260808 08:27:41.875066 16761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.194:42205 every 8 connection(s)
I20260808 08:27:41.876005 16544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
I20260808 08:27:41.879684 16764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:41.878106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.194:36243)
0808 08:27:41.878578 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:41.878986 (+ 408us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":138,"threads_started":1}
W20260808 08:27:41.880119 16762 heartbeater.cc:644] Failed to heartbeat to 127.22.235.254:46193 (0 consecutive failures): Network error: Failed to ping master at 127.22.235.254:46193: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
I20260808 08:27:41.880690 16764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:41.880479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.194:39391)
0808 08:27:41.880574 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:41.880637 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 08:27:41.881217 16764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:41.881060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.194:47123)
0808 08:27:41.881136 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:41.881173 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 08:27:41.881383 16762 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 08:27:41.883168 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 16544
I20260808 08:27:41.883555 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.195:37593
--local_ip_for_outbound_sockets=127.22.235.195
--tserver_master_addrs=127.22.235.254:46193
--webserver_port=43719
--webserver_interface=127.22.235.195
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab
--principal=kudu/127.22.235.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:40861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:27:41.995196 16766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:27:41.995435 16766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:27:41.995474 16766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:27:41.999300 16766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:27:41.999351 16766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:27:41.999406 16766 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.195
I20260808 08:27:42.004410 16766 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:40861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab
--principal=kudu/127.22.235.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.195:37593
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.235.195
--webserver_port=43719
--tserver_master_addrs=127.22.235.254:46193
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16766
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.195
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:27:42.005394 16766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:27:42.006359 16766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:27:42.015775 16777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:27:42.015799 16779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:27:42.015841 16766 server_base.cc:1034] running on GCE node
W20260808 08:27:42.015775 16783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:27:42.016471 16766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:27:42.017112 16766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:27:42.018260 16766 hybrid_clock.cc:648] HybridClock initialized: now 1786177662018235 us; error 39 us; skew 500 ppm
Aug 08 08:27:42 dist-test-slave-f7th krb5kdc[14368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177662, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:27:42.021020 16766 init.cc:377] Logged in from keytab as kudu/127.22.235.195@KRBTEST.COM (short username kudu)
I20260808 08:27:42.021826 16766 webserver.cc:460] Webserver started at http://127.22.235.195:43719/ using document root <none> and password file <none>
I20260808 08:27:42.022353 16766 fs_manager.cc:359] Metadata directory not provided
I20260808 08:27:42.022415 16766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:27:42.026091 16766 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:27:42.028038 16793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:42.029258 16766 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:27:42.029361 16766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-2/wal
uuid: "a8037a31939547ffa83bb358dc39307e"
format_stamp: "Formatted at 2026-08-08 08:27:39 on dist-test-slave-f7th"
server_key: "35da105d6b2ff23bd8af75eca5fed51a"
server_key_iv: "2ec23ad4c3b20e937be4c244ffdfe840"
server_key_version: "encryptionkey@0"
I20260808 08:27:42.029873 16766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:27:42.065475 16766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:27:42.068228 16766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:27:42.068385 16766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:27:42.068974 16766 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:27:42.069792 16766 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:27:42.069839 16766 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:42.069878 16766 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:27:42.069900 16766 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:42.078580 16766 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.195:37593
I20260808 08:27:42.078670 16963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.195:37593 every 8 connection(s)
I20260808 08:27:42.079574 16766 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
I20260808 08:27:42.080725 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 16766
I20260808 08:27:42.083285 16967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:42.081724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.195:55011)
0808 08:27:42.082177 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:42.082588 (+ 411us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":113,"threads_started":1}
W20260808 08:27:42.083717 16964 heartbeater.cc:644] Failed to heartbeat to 127.22.235.254:46193 (0 consecutive failures): Network error: Failed to ping master at 127.22.235.254:46193: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
I20260808 08:27:42.084288 16967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:42.084086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.195:34829)
0808 08:27:42.084187 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:42.084234 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 08:27:42.084831 16967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:42.084656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.195:47279)
0808 08:27:42.084744 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:42.084785 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 08:27:42.084980 16964 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 08:27:42.769172 16969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:42.768734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.193:35633)
0808 08:27:42.768999 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:42.769056 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":76,"threads_started":1}
I20260808 08:27:42.882288 16970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:42.881882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.194:36919)
0808 08:27:42.882134 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:42.882188 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":81,"threads_started":1}
I20260808 08:27:43.085850 16972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:43.085471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.195:48571)
0808 08:27:43.085697 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:43.085751 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":70,"threads_started":1}
I20260808 08:27:43.380448 16973 raft_consensus.cc:482] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 08:27:43.380667 16973 raft_consensus.cc:504] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8037a31939547ffa83bb358dc39307e" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37593 } } peers { permanent_uuid: "a6d6813abd514f13a3e5880174713c3a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 42205 } } peers { permanent_uuid: "b2e3f9bc786849869ce8e2432df29b53" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44193 } }
I20260808 08:27:43.381728 16973 leader_election.cc:288] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a8037a31939547ffa83bb358dc39307e (127.22.235.195:37593), a6d6813abd514f13a3e5880174713c3a (127.22.235.194:42205)
Aug 08 08:27:43 dist-test-slave-f7th krb5kdc[14368](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177661, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for kudu/127.22.235.194@KRBTEST.COM
Aug 08 08:27:43 dist-test-slave-f7th krb5kdc[14368](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177661, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for kudu/127.22.235.195@KRBTEST.COM
I20260808 08:27:43.397928 16976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:43.381916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.194:42205 (local address 127.22.235.193:43125)
0808 08:27:43.382335 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:43.382346 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:43.382419 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:43.384399 (+ 1980us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:27:43.384406 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:27:43.385054 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:27:43.385862 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:43.385885 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:43.387795 (+ 1910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:43.387798 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:43.389285 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:43.389298 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:43.390231 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:43.390235 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:43.390497 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:43.391202 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:27:43.391225 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:27:43.393030 (+ 1805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:27:43.395326 (+ 2296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:43.395333 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:43.395349 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:43.395839 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:43.396225 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:43.396229 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:43.396231 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:43.396366 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:43.397009 (+ 643us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:27:43.397015 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:27:43.397253 (+ 238us) client_negotiation.cc:769] Sending connection context
0808 08:27:43.397408 (+ 155us) client_negotiation.cc:240] Negotiation successful
0808 08:27:43.397736 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"mutex_wait_us":219,"spinlock_wait_cycles":42240,"thread_start_us":47,"threads_started":1}
I20260808 08:27:43.397984 16979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:43.382111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:56297 (local address 127.22.235.195:37593)
0808 08:27:43.382528 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:43.382547 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:43.382584 (+ 37us) server_negotiation.cc:407] Connection header received
0808 08:27:43.383457 (+ 873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:43.383467 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:43.383927 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:43.384262 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:27:43.386066 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:43.387632 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:43.389396 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:43.390108 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:43.390273 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:43.393571 (+ 3298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:27:43.393578 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:27:43.393593 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:27:43.393627 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:27:43.395261 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:43.395989 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:43.395996 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:43.396002 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:43.396140 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:43.396474 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:43.396477 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:43.396479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:43.396896 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:27:43.397029 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:43.397563 (+ 534us) server_negotiation.cc:299] Negotiation successful
0808 08:27:43.397818 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":104,"threads_started":1}
I20260808 08:27:43.398018 16978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:43.382111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:43125 (local address 127.22.235.194:42205)
0808 08:27:43.382506 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:43.382536 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:43.382584 (+ 48us) server_negotiation.cc:407] Connection header received
0808 08:27:43.383457 (+ 873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:43.383467 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:43.383927 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:43.384262 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:27:43.386068 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:43.387632 (+ 1564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:43.389395 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:43.390108 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:43.390269 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:43.393266 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:27:43.393273 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:27:43.393290 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:27:43.393321 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:27:43.395210 (+ 1889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:43.395967 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:43.395973 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:43.395979 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:43.396129 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:43.396465 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:43.396469 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:43.396470 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:43.396895 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:27:43.397024 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:43.397556 (+ 532us) server_negotiation.cc:299] Negotiation successful
0808 08:27:43.397821 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":68,"threads_started":1}
I20260808 08:27:43.398828 16975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:43.381916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.195:37593 (local address 127.22.235.193:56297)
0808 08:27:43.382225 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:43.382241 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:43.382353 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:43.384399 (+ 2046us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:27:43.384406 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:27:43.385054 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:27:43.385862 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:43.385885 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:43.387795 (+ 1910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:43.387798 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:43.389285 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:43.389298 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:43.390231 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:43.390235 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:43.390497 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:43.391211 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:27:43.391230 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:27:43.393377 (+ 2147us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:27:43.395349 (+ 1972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:43.395351 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:43.395353 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:43.395839 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:43.396245 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:43.396249 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:43.396251 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:43.396371 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:43.397013 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:27:43.397016 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:27:43.397248 (+ 232us) client_negotiation.cc:769] Sending connection context
0808 08:27:43.397408 (+ 160us) client_negotiation.cc:240] Negotiation successful
0808 08:27:43.397821 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":445,"spinlock_wait_cycles":28544,"thread_start_us":81,"threads_started":1}
W20260808 08:27:43.400570 16372 leader_election.cc:341] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a6d6813abd514f13a3e5880174713c3a (127.22.235.194:42205): Not found: Tablet not found: ed717835f2824ad584c6319d2636a81c
W20260808 08:27:43.400928 16369 leader_election.cc:341] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a8037a31939547ffa83bb358dc39307e (127.22.235.195:37593): Not found: Tablet not found: ed717835f2824ad584c6319d2636a81c
I20260808 08:27:43.401005 16369 leader_election.cc:302] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e3f9bc786849869ce8e2432df29b53; no voters: a6d6813abd514f13a3e5880174713c3a, a8037a31939547ffa83bb358dc39307e
I20260808 08:27:43.401230 16973 raft_consensus.cc:2726] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 08:27:43.770120 16975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:43.769845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.193:35685)
0808 08:27:43.769979 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:43.770032 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 08:27:43.883399 16980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:43.883000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.194:59671)
0808 08:27:43.883256 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:43.883305 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":106,"threads_started":1}
I20260808 08:27:44.086892 16982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:44.086516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.195:59687)
0808 08:27:44.086735 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:44.086797 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":73,"threads_started":1}
I20260808 08:27:44.771277 16983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:44.770856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.193:36735)
0808 08:27:44.771121 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:44.771179 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":82,"threads_started":1}
I20260808 08:27:44.884459 16984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:44.884072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.194:47971)
0808 08:27:44.884309 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:44.884364 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":72,"threads_started":1}
I20260808 08:27:45.087970 16985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:45.087574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.195:53583)
0808 08:27:45.087804 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:45.087855 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":78,"threads_started":1}
I20260808 08:27:45.505651 16989 raft_consensus.cc:482] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 08:27:45.505793 16989 raft_consensus.cc:504] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8037a31939547ffa83bb358dc39307e" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37593 } } peers { permanent_uuid: "a6d6813abd514f13a3e5880174713c3a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 42205 } } peers { permanent_uuid: "b2e3f9bc786849869ce8e2432df29b53" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44193 } }
I20260808 08:27:45.506248 16989 leader_election.cc:288] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a8037a31939547ffa83bb358dc39307e (127.22.235.195:37593), a6d6813abd514f13a3e5880174713c3a (127.22.235.194:42205)
W20260808 08:27:45.507084 16372 leader_election.cc:341] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a6d6813abd514f13a3e5880174713c3a (127.22.235.194:42205): Not found: Tablet not found: ed717835f2824ad584c6319d2636a81c
W20260808 08:27:45.507201 16369 leader_election.cc:341] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a8037a31939547ffa83bb358dc39307e (127.22.235.195:37593): Not found: Tablet not found: ed717835f2824ad584c6319d2636a81c
I20260808 08:27:45.507282 16369 leader_election.cc:302] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e3f9bc786849869ce8e2432df29b53; no voters: a6d6813abd514f13a3e5880174713c3a, a8037a31939547ffa83bb358dc39307e
I20260808 08:27:45.507419 16989 raft_consensus.cc:2726] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 08:27:45.772502 16990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:45.772084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.193:41699)
0808 08:27:45.772343 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:45.772399 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":82,"threads_started":1}
I20260808 08:27:45.885604 16991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:45.885145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.194:55033)
0808 08:27:45.885429 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:45.885488 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":76,"threads_started":1}
I20260808 08:27:46.089097 16992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:46.088714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.195:40523)
0808 08:27:46.088952 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:46.089008 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":70,"threads_started":1}
I20260808 08:27:46.773897 16993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:46.773455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.193:37265)
0808 08:27:46.773748 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:46.773800 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":75,"threads_started":1}
I20260808 08:27:46.886801 16994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:46.886377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.194:39595)
0808 08:27:46.886634 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:46.886689 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":82,"threads_started":1}
I20260808 08:27:47.090227 16995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:47.089803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.195:57369)
0808 08:27:47.090070 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:47.090130 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":78,"threads_started":1}
I20260808 08:27:47.774971 16997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:47.774599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.193:51775)
0808 08:27:47.774825 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:47.774877 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":72,"threads_started":1}
I20260808 08:27:47.887888 16998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:47.887510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.194:42597)
0808 08:27:47.887736 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:47.887792 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":72,"threads_started":1}
I20260808 08:27:48.091432 16999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:48.091043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.195:50935)
0808 08:27:48.091281 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:48.091336 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":78,"threads_started":1}
I20260808 08:27:48.324842 17000 raft_consensus.cc:482] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 08:27:48.324988 17000 raft_consensus.cc:504] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8037a31939547ffa83bb358dc39307e" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37593 } } peers { permanent_uuid: "a6d6813abd514f13a3e5880174713c3a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 42205 } } peers { permanent_uuid: "b2e3f9bc786849869ce8e2432df29b53" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44193 } }
I20260808 08:27:48.325395 17000 leader_election.cc:288] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a8037a31939547ffa83bb358dc39307e (127.22.235.195:37593), a6d6813abd514f13a3e5880174713c3a (127.22.235.194:42205)
W20260808 08:27:48.326252 16369 leader_election.cc:341] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a8037a31939547ffa83bb358dc39307e (127.22.235.195:37593): Not found: Tablet not found: ed717835f2824ad584c6319d2636a81c
W20260808 08:27:48.326401 16372 leader_election.cc:341] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a6d6813abd514f13a3e5880174713c3a (127.22.235.194:42205): Not found: Tablet not found: ed717835f2824ad584c6319d2636a81c
I20260808 08:27:48.326473 16372 leader_election.cc:302] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e3f9bc786849869ce8e2432df29b53; no voters: a6d6813abd514f13a3e5880174713c3a, a8037a31939547ffa83bb358dc39307e
I20260808 08:27:48.326615 17000 raft_consensus.cc:2726] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 08:27:48.776093 17002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:48.775679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.193:53617)
0808 08:27:48.775940 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:48.775995 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":96,"threads_started":1}
I20260808 08:27:48.889163 17003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:48.888585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.194:57653)
0808 08:27:48.888977 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:48.889060 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":75,"threads_started":1}
I20260808 08:27:49.092650 17004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:49.092139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.195:37057)
0808 08:27:49.092450 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:49.092521 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":73,"threads_started":1}
I20260808 08:27:49.777233 17005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:49.776818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.193:59377)
0808 08:27:49.777080 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:49.777135 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":80,"threads_started":1}
I20260808 08:27:49.890154 17006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:49.889794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.194:41931)
0808 08:27:49.890011 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:49.890060 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":75,"threads_started":1}
I20260808 08:27:50.093693 17008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:50.093274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.195:56611)
0808 08:27:50.093533 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:50.093587 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":75,"threads_started":1}
I20260808 08:27:50.778333 17009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:50.777922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.193:38547)
0808 08:27:50.778182 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:50.778234 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":95,"threads_started":1}
I20260808 08:27:50.891233 17013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:50.890807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.194:42041)
0808 08:27:50.891079 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:50.891139 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":78,"threads_started":1}
I20260808 08:27:51.094815 17014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:51.094354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.195:39875)
0808 08:27:51.094663 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:51.094721 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":108,"threads_started":1}
I20260808 08:27:51.779570 17016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:51.779185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.193:56861)
0808 08:27:51.779419 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:51.779471 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":77,"threads_started":1}
I20260808 08:27:51.892375 17017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:51.891946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.194:52449)
0808 08:27:51.892210 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:51.892266 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":88,"threads_started":1}
I20260808 08:27:52.095983 17018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:52.095572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.195:35321)
0808 08:27:52.095837 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:52.095893 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":93,"threads_started":1}
I20260808 08:27:52.325331 17019 raft_consensus.cc:482] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 08:27:52.325471 17019 raft_consensus.cc:504] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8037a31939547ffa83bb358dc39307e" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37593 } } peers { permanent_uuid: "a6d6813abd514f13a3e5880174713c3a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 42205 } } peers { permanent_uuid: "b2e3f9bc786849869ce8e2432df29b53" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44193 } }
I20260808 08:27:52.325862 17019 leader_election.cc:288] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a8037a31939547ffa83bb358dc39307e (127.22.235.195:37593), a6d6813abd514f13a3e5880174713c3a (127.22.235.194:42205)
W20260808 08:27:52.326671 16369 leader_election.cc:341] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a8037a31939547ffa83bb358dc39307e (127.22.235.195:37593): Not found: Tablet not found: ed717835f2824ad584c6319d2636a81c
W20260808 08:27:52.326797 16372 leader_election.cc:341] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a6d6813abd514f13a3e5880174713c3a (127.22.235.194:42205): Not found: Tablet not found: ed717835f2824ad584c6319d2636a81c
I20260808 08:27:52.326887 16372 leader_election.cc:302] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e3f9bc786849869ce8e2432df29b53; no voters: a6d6813abd514f13a3e5880174713c3a, a8037a31939547ffa83bb358dc39307e
I20260808 08:27:52.327067 17019 raft_consensus.cc:2726] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 08:27:52.780725 17020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:52.780272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.193:46835)
0808 08:27:52.780535 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:52.780596 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":97,"threads_started":1}
I20260808 08:27:52.893440 17021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:52.893041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.194:45023)
0808 08:27:52.893295 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:52.893346 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":72,"threads_started":1}
I20260808 08:27:53.097116 17023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:53.096720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.195:46349)
0808 08:27:53.096969 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:53.097022 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":89,"threads_started":1}
I20260808 08:27:53.781893 17024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:53.781444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.193:50337)
0808 08:27:53.781731 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:53.781791 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":93,"threads_started":1}
I20260808 08:27:53.894492 17028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:53.894089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.194:45309)
0808 08:27:53.894343 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:53.894395 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":96,"threads_started":1}
I20260808 08:27:54.098188 17029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:54.097775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.195:41677)
0808 08:27:54.098033 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:54.098091 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":74,"threads_started":1}
I20260808 08:27:54.783072 17030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:54.782631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.193:51327)
0808 08:27:54.782911 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:54.782973 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":76,"threads_started":1}
I20260808 08:27:54.895602 17031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:54.895181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.194:51623)
0808 08:27:54.895435 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:54.895501 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":76,"threads_started":1}
I20260808 08:27:55.099264 17032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:55.098861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.195:52821)
0808 08:27:55.099117 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:55.099171 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":78,"threads_started":1}
I20260808 08:27:55.784237 17033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:55.783810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.193:36843)
0808 08:27:55.784087 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:55.784142 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":94,"threads_started":1}
I20260808 08:27:55.896703 17034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:55.896292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.194:46365)
0808 08:27:55.896523 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:55.896577 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":71,"threads_started":1}
I20260808 08:27:56.100409 17036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:56.100006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.195:54791)
0808 08:27:56.100257 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:56.100312 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":82,"threads_started":1}
I20260808 08:27:56.711875 17037 raft_consensus.cc:482] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 08:27:56.712016 17037 raft_consensus.cc:504] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8037a31939547ffa83bb358dc39307e" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37593 } } peers { permanent_uuid: "a6d6813abd514f13a3e5880174713c3a" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 42205 } } peers { permanent_uuid: "b2e3f9bc786849869ce8e2432df29b53" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44193 } }
I20260808 08:27:56.712460 17037 leader_election.cc:288] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a8037a31939547ffa83bb358dc39307e (127.22.235.195:37593), a6d6813abd514f13a3e5880174713c3a (127.22.235.194:42205)
W20260808 08:27:56.713255 16369 leader_election.cc:341] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a8037a31939547ffa83bb358dc39307e (127.22.235.195:37593): Not found: Tablet not found: ed717835f2824ad584c6319d2636a81c
W20260808 08:27:56.713371 16372 leader_election.cc:341] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a6d6813abd514f13a3e5880174713c3a (127.22.235.194:42205): Not found: Tablet not found: ed717835f2824ad584c6319d2636a81c
I20260808 08:27:56.713466 16372 leader_election.cc:302] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2e3f9bc786849869ce8e2432df29b53; no voters: a6d6813abd514f13a3e5880174713c3a, a8037a31939547ffa83bb358dc39307e
I20260808 08:27:56.713657 17037 raft_consensus.cc:2726] T ed717835f2824ad584c6319d2636a81c P b2e3f9bc786849869ce8e2432df29b53 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 08:27:56.785578 17038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:56.785181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.193:42423)
0808 08:27:56.785427 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:56.785479 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":100,"threads_started":1}
I20260808 08:27:56.897913 17040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:56.897514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.194:44899)
0808 08:27:56.897765 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:56.897819 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":90,"threads_started":1}
I20260808 08:27:57.082688 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 16291
I20260808 08:27:57.101732 17041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:57.101307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46193 (local address 127.22.235.195:59043)
0808 08:27:57.101574 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:57.101633 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":64,"threads_started":1}
I20260808 08:27:57.106266 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 16544
I20260808 08:27:57.121204 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 16766
2026-08-08T08:27:57Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19737 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 08:27:57 dist-test-slave-f7th krb5kdc[17045](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 08:27:57 dist-test-slave-f7th krb5kdc[17045](info): set up 2 sockets
Aug 08 08:27:57 dist-test-slave-f7th krb5kdc[17045](info): commencing operation
krb5kdc: starting...
W20260808 08:27:58.289819 23471 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Aug 08 08:27:58 dist-test-slave-f7th krb5kdc[17045](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T08:27:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:27:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.235.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
I20260808 08:27:58.407343 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:46859
--webserver_interface=127.22.235.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:41847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:46859
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:27:58.517272 17063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:27:58.517513 17063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:27:58.517550 17063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:27:58.521369 17063 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 08:27:58.521414 17063 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 08:27:58.521428 17063 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 08:27:58.521440 17063 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:27:58.521456 17063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:27:58.521487 17063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:27:58.521507 17063 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:27:58.521520 17063 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:27:58.526912 17063 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:41847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:46859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:46859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.235.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17063
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:27:58.527868 17063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:27:58.528829 17063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:27:58.536944 17070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:27:58.536934 17073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:27:58.536978 17071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:27:58.537067 17063 server_base.cc:1034] running on GCE node
I20260808 08:27:58.537703 17063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:27:58.538713 17063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:27:58.539881 17063 hybrid_clock.cc:648] HybridClock initialized: now 1786177678539856 us; error 36 us; skew 500 ppm
Aug 08 08:27:58 dist-test-slave-f7th krb5kdc[17045](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177678, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:27:58.542497 17063 init.cc:377] Logged in from keytab as kudu/127.22.235.254@KRBTEST.COM (short username kudu)
I20260808 08:27:58.543344 17063 webserver.cc:460] Webserver started at http://127.22.235.254:34107/ using document root <none> and password file <none>
I20260808 08:27:58.543853 17063 fs_manager.cc:359] Metadata directory not provided
I20260808 08:27:58.543918 17063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:27:58.544123 17063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:27:58.546104 17063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "955105d3eda549ddb05259d3c312c676"
format_stamp: "Formatted at 2026-08-08 08:27:58 on dist-test-slave-f7th"
server_key: "14611b503cf030b60a6e621bfa39d2b0"
server_key_iv: "b669d78e0ca36526072a5faa4123b6f3"
server_key_version: "encryptionkey@0"
I20260808 08:27:58.546625 17063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "955105d3eda549ddb05259d3c312c676"
format_stamp: "Formatted at 2026-08-08 08:27:58 on dist-test-slave-f7th"
server_key: "14611b503cf030b60a6e621bfa39d2b0"
server_key_iv: "b669d78e0ca36526072a5faa4123b6f3"
server_key_version: "encryptionkey@0"
I20260808 08:27:58.550339 17063 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260808 08:27:58.552799 17084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:58.553985 17063 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260808 08:27:58.554116 17063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "955105d3eda549ddb05259d3c312c676"
format_stamp: "Formatted at 2026-08-08 08:27:58 on dist-test-slave-f7th"
server_key: "14611b503cf030b60a6e621bfa39d2b0"
server_key_iv: "b669d78e0ca36526072a5faa4123b6f3"
server_key_version: "encryptionkey@0"
I20260808 08:27:58.554208 17063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:27:58.569947 17063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:27:58.572665 17063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:27:58.572836 17063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:27:58.579545 17063 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.254:46859
I20260808 08:27:58.579799 17177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.254:46859 every 8 connection(s)
I20260808 08:27:58.580528 17063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:27:58.582616 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 17063
I20260808 08:27:58.582726 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:27:58.582994 23471 external_mini_cluster.cc:1428] Setting key 3e4b317a16da1a9c20444831d013f89a
I20260808 08:27:58.583464 17178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:27:58.590407 17178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676: Bootstrap starting.
Aug 08 08:27:58 dist-test-slave-f7th krb5kdc[17045](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:27:58.592685 17178 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676: Neither blocks nor log segments found. Creating new log.
I20260808 08:27:58.593550 17178 log.cc:824] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676: Log is configured to *not* fsync() on all Append() calls
I20260808 08:27:58.595690 17178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676: No bootstrap required, opened a new log
I20260808 08:27:58.597319 17181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:58.584389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51204 (local address 127.22.235.254:46859)
0808 08:27:58.584826 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:58.584846 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:58.584890 (+ 44us) server_negotiation.cc:407] Connection header received
0808 08:27:58.585724 (+ 834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:58.585750 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:58.586175 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:58.586537 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:27:58.587217 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:58.588686 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:58.589514 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:58.590178 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:58.590333 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:58.592664 (+ 2331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:27:58.592671 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:27:58.592687 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:27:58.592720 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:27:58.594404 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:58.594804 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:58.594810 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:58.594815 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:58.594952 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:58.595190 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:58.595193 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:58.595194 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:58.595670 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:27:58.595860 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:58.596105 (+ 245us) server_negotiation.cc:299] Negotiation successful
0808 08:27:58.596389 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":140,"threads_started":1}
I20260808 08:27:58.599738 17178 raft_consensus.cc:348] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "955105d3eda549ddb05259d3c312c676" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46859 } }
I20260808 08:27:58.599938 17178 raft_consensus.cc:374] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:27:58.599987 17178 raft_consensus.cc:729] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955105d3eda549ddb05259d3c312c676, State: Initialized, Role: FOLLOWER
I20260808 08:27:58.600363 17178 consensus_queue.cc:260] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "955105d3eda549ddb05259d3c312c676" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46859 } }
I20260808 08:27:58.600482 17178 raft_consensus.cc:388] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:27:58.600534 17178 raft_consensus.cc:482] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:27:58.600602 17178 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:27:58.601406 17178 raft_consensus.cc:504] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "955105d3eda549ddb05259d3c312c676" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46859 } }
I20260808 08:27:58.601698 17178 leader_election.cc:302] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 955105d3eda549ddb05259d3c312c676; no voters:
I20260808 08:27:58.601979 17178 leader_election.cc:288] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:27:58.602099 17183 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:27:58.602319 17183 raft_consensus.cc:686] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [term 1 LEADER]: Becoming Leader. State: Replica: 955105d3eda549ddb05259d3c312c676, State: Running, Role: LEADER
I20260808 08:27:58.602644 17183 consensus_queue.cc:237] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "955105d3eda549ddb05259d3c312c676" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46859 } }
I20260808 08:27:58.603394 17178 sys_catalog.cc:564] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:27:58.604763 17185 sys_catalog.cc:455] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "955105d3eda549ddb05259d3c312c676" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "955105d3eda549ddb05259d3c312c676" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46859 } } }
I20260808 08:27:58.606375 17185 sys_catalog.cc:458] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [sys.catalog]: This master's current role is: LEADER
I20260808 08:27:58.604935 17188 sys_catalog.cc:455] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 955105d3eda549ddb05259d3c312c676. Latest consensus state: current_term: 1 leader_uuid: "955105d3eda549ddb05259d3c312c676" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "955105d3eda549ddb05259d3c312c676" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46859 } } }
I20260808 08:27:58.606729 17188 sys_catalog.cc:458] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [sys.catalog]: This master's current role is: LEADER
W20260808 08:27:58.608706 17211 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 08:27:58.608795 17211 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 08:27:58.608915 17213 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:27:58.609848 17213 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:27:58.615796 17213 catalog_manager.cc:1347] Generated new cluster ID: 01abe1afe9464a23bca90e7f4fed11c1
I20260808 08:27:58.615903 17213 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:27:58.627285 17213 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:27:58.628269 17213 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:27:58.632781 17213 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676: Generated new TSK 0
I20260808 08:27:58.633484 17213 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.235.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab.
Entry for principal kudu/127.22.235.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab.
I20260808 08:27:58.668772 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.193:0
--local_ip_for_outbound_sockets=127.22.235.193
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:46859
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab
--principal=kudu/127.22.235.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:41847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:27:58.780448 17222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:27:58.780745 17222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:27:58.780810 17222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:27:58.784871 17222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:27:58.784925 17222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:27:58.784991 17222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.193
I20260808 08:27:58.790172 17222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:41847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab
--principal=kudu/127.22.235.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:46859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.193
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:27:58.791165 17222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:27:58.792372 17222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 08:27:58.801697 17222 server_base.cc:1034] running on GCE node
W20260808 08:27:58.801584 17231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:27:58.801591 17234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:27:58.801771 17229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:27:58.802327 17222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:27:58.802950 17222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:27:58.804100 17222 hybrid_clock.cc:648] HybridClock initialized: now 1786177678804087 us; error 24 us; skew 500 ppm
Aug 08 08:27:58 dist-test-slave-f7th krb5kdc[17045](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177678, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:27:58.806804 17222 init.cc:377] Logged in from keytab as kudu/127.22.235.193@KRBTEST.COM (short username kudu)
I20260808 08:27:58.807678 17222 webserver.cc:460] Webserver started at http://127.22.235.193:32955/ using document root <none> and password file <none>
I20260808 08:27:58.808212 17222 fs_manager.cc:359] Metadata directory not provided
I20260808 08:27:58.808279 17222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:27:58.808480 17222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:27:58.810474 17222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-0/data/instance:
uuid: "5e6519a7db6245f9b459ee5905af464b"
format_stamp: "Formatted at 2026-08-08 08:27:58 on dist-test-slave-f7th"
server_key: "19f824c19d97b739b738edbccf4247cd"
server_key_iv: "7d3c06265e9b909dba1c2f01ae18a614"
server_key_version: "encryptionkey@0"
I20260808 08:27:58.810989 17222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance:
uuid: "5e6519a7db6245f9b459ee5905af464b"
format_stamp: "Formatted at 2026-08-08 08:27:58 on dist-test-slave-f7th"
server_key: "19f824c19d97b739b738edbccf4247cd"
server_key_iv: "7d3c06265e9b909dba1c2f01ae18a614"
server_key_version: "encryptionkey@0"
I20260808 08:27:58.814586 17222 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 08:27:58.817167 17246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:58.818315 17222 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:27:58.818423 17222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-0/wal
uuid: "5e6519a7db6245f9b459ee5905af464b"
format_stamp: "Formatted at 2026-08-08 08:27:58 on dist-test-slave-f7th"
server_key: "19f824c19d97b739b738edbccf4247cd"
server_key_iv: "7d3c06265e9b909dba1c2f01ae18a614"
server_key_version: "encryptionkey@0"
I20260808 08:27:58.818513 17222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:27:58.834358 17222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:27:58.837021 17222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:27:58.837193 17222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:27:58.837759 17222 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:27:58.838585 17222 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:27:58.838634 17222 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:58.838688 17222 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:27:58.838716 17222 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:58.847515 17222 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.193:45483
I20260808 08:27:58.847541 17417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.193:45483 every 8 connection(s)
I20260808 08:27:58.848430 17222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
I20260808 08:27:58.855286 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 17222
I20260808 08:27:58.855414 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance
I20260808 08:27:58.855741 23471 external_mini_cluster.cc:1428] Setting key 33d20eebb7bd9d139d12c796e5686de7
Aug 08 08:27:58 dist-test-slave-f7th krb5kdc[17045](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177678, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:27:58.861853 17181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:58.850257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:54383 (local address 127.22.235.254:46859)
0808 08:27:58.850386 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:58.850390 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:58.851213 (+ 823us) server_negotiation.cc:407] Connection header received
0808 08:27:58.852409 (+ 1196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:58.852412 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:58.852459 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:58.852536 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:27:58.853684 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:58.854204 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:58.855319 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:58.855654 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:58.855765 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:58.858421 (+ 2656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:27:58.858425 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:27:58.858427 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:27:58.858455 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:27:58.859800 (+ 1345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:58.860358 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:58.860360 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:58.860361 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:58.860428 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:58.860808 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:58.860810 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:58.860812 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:58.860982 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:27:58.861063 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:58.861630 (+ 567us) server_negotiation.cc:299] Negotiation successful
0808 08:27:58.861732 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 08:27:58.863001 17423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:58.850612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.193:54383)
0808 08:27:58.851060 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:58.851103 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:58.852220 (+ 1117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:58.852712 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:27:58.852719 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:27:58.853085 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:27:58.853560 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:58.853575 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:58.854297 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:58.854300 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:58.855195 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:58.855205 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:58.855805 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:58.855809 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:58.855983 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:58.856588 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:27:58.856609 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:27:58.858246 (+ 1637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:27:58.859911 (+ 1665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:58.859917 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:58.859933 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:58.860273 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:58.860516 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:58.860519 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:58.860522 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:58.860728 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:58.861088 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:27:58.861094 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:27:58.861396 (+ 302us) client_negotiation.cc:769] Sending connection context
0808 08:27:58.861599 (+ 203us) client_negotiation.cc:240] Negotiation successful
0808 08:27:58.861927 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":105,"threads_started":1}
I20260808 08:27:58.864205 17419 heartbeater.cc:347] Connected to a master server at 127.22.235.254:46859
I20260808 08:27:58.864432 17419 heartbeater.cc:464] Registering TS with master...
I20260808 08:27:58.864953 17419 heartbeater.cc:511] Master 127.22.235.254:46859 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.22.235.194@KRBTEST.COM; defaulting to no policy
I20260808 08:27:58.866456 17117 ts_manager.cc:194] Registered new tserver with Master: 5e6519a7db6245f9b459ee5905af464b (127.22.235.193:45483)
I20260808 08:27:58.867604 17117 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.193@KRBTEST.COM'} at 127.22.235.193:54383
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab.
Entry for principal kudu/127.22.235.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab.
I20260808 08:27:58.883688 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.194:0
--local_ip_for_outbound_sockets=127.22.235.194
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:46859
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab
--principal=kudu/127.22.235.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:41847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:27:59.007141 17427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:27:59.007503 17427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:27:59.007588 17427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:27:59.012761 17427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:27:59.012820 17427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:27:59.012907 17427 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.194
I20260808 08:27:59.018249 17427 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:41847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab
--principal=kudu/127.22.235.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:46859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.194
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:27:59.019323 17427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:27:59.020582 17427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:27:59.030143 17435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:27:59.030167 17433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:27:59.030161 17432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:27:59.030318 17427 server_base.cc:1034] running on GCE node
I20260808 08:27:59.030797 17427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:27:59.031388 17427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:27:59.032546 17427 hybrid_clock.cc:648] HybridClock initialized: now 1786177679032519 us; error 39 us; skew 500 ppm
Aug 08 08:27:59 dist-test-slave-f7th krb5kdc[17045](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177679, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:27:59.035080 17427 init.cc:377] Logged in from keytab as kudu/127.22.235.194@KRBTEST.COM (short username kudu)
I20260808 08:27:59.035970 17427 webserver.cc:460] Webserver started at http://127.22.235.194:43027/ using document root <none> and password file <none>
I20260808 08:27:59.036528 17427 fs_manager.cc:359] Metadata directory not provided
I20260808 08:27:59.036597 17427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:27:59.036831 17427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:27:59.038792 17427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-1/data/instance:
uuid: "23f05ff8a1b04bb0a5a11717faec4639"
format_stamp: "Formatted at 2026-08-08 08:27:59 on dist-test-slave-f7th"
server_key: "d5d81801df520fb0567eb4182667b79a"
server_key_iv: "a19e140047c133a8959da581b505119c"
server_key_version: "encryptionkey@0"
I20260808 08:27:59.039322 17427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance:
uuid: "23f05ff8a1b04bb0a5a11717faec4639"
format_stamp: "Formatted at 2026-08-08 08:27:59 on dist-test-slave-f7th"
server_key: "d5d81801df520fb0567eb4182667b79a"
server_key_iv: "a19e140047c133a8959da581b505119c"
server_key_version: "encryptionkey@0"
I20260808 08:27:59.042979 17427 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 08:27:59.045310 17445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:59.046425 17427 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:27:59.046552 17427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-1/wal
uuid: "23f05ff8a1b04bb0a5a11717faec4639"
format_stamp: "Formatted at 2026-08-08 08:27:59 on dist-test-slave-f7th"
server_key: "d5d81801df520fb0567eb4182667b79a"
server_key_iv: "a19e140047c133a8959da581b505119c"
server_key_version: "encryptionkey@0"
I20260808 08:27:59.046644 17427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:27:59.062909 17427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:27:59.065555 17427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:27:59.065729 17427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:27:59.066294 17427 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:27:59.067132 17427 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:27:59.067181 17427 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:59.067235 17427 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:27:59.067263 17427 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:59.075743 17427 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.194:39403
I20260808 08:27:59.075769 17631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.194:39403 every 8 connection(s)
I20260808 08:27:59.076633 17427 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
I20260808 08:27:59.082293 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 17427
I20260808 08:27:59.082407 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance
I20260808 08:27:59.082707 23471 external_mini_cluster.cc:1428] Setting key fff2322bf578259a7c549e320c4d9db0
Aug 08 08:27:59 dist-test-slave-f7th krb5kdc[17045](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177679, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.194@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:27:59.090386 17181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:59.078384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:38351 (local address 127.22.235.254:46859)
0808 08:27:59.078524 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:59.078527 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:59.079333 (+ 806us) server_negotiation.cc:407] Connection header received
0808 08:27:59.080568 (+ 1235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:59.080572 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:59.080660 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:59.080734 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:27:59.081859 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:59.082554 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:59.083936 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:59.084227 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:59.084332 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:59.086800 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:27:59.086803 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:27:59.086805 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:27:59.086828 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:27:59.088262 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:59.088791 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:59.088795 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:59.088796 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:59.088890 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:59.089199 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:59.089203 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:59.089205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:59.089475 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:27:59.089594 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:59.090087 (+ 493us) server_negotiation.cc:299] Negotiation successful
0808 08:27:59.090218 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 08:27:59.091120 17635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:59.078755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.194:38351)
0808 08:27:59.079197 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:59.079236 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:59.080362 (+ 1126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:59.080911 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:27:59.080917 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:27:59.081260 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:27:59.081719 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:59.081733 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:59.082684 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:59.082688 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:59.083813 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:59.083825 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:59.084373 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:59.084376 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:59.084518 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:59.085082 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:27:59.085099 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:27:59.086638 (+ 1539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:27:59.088373 (+ 1735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:59.088379 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:59.088391 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:59.088701 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:59.088977 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:59.088979 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:59.088981 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:59.089112 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:59.089600 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:27:59.089605 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:27:59.089841 (+ 236us) client_negotiation.cc:769] Sending connection context
0808 08:27:59.090008 (+ 167us) client_negotiation.cc:240] Negotiation successful
0808 08:27:59.090265 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":116,"threads_started":1}
I20260808 08:27:59.092238 17633 heartbeater.cc:347] Connected to a master server at 127.22.235.254:46859
I20260808 08:27:59.092509 17633 heartbeater.cc:464] Registering TS with master...
I20260808 08:27:59.093139 17633 heartbeater.cc:511] Master 127.22.235.254:46859 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.22.235.195@KRBTEST.COM; defaulting to no policy
I20260808 08:27:59.094264 17117 ts_manager.cc:194] Registered new tserver with Master: 23f05ff8a1b04bb0a5a11717faec4639 (127.22.235.194:39403)
I20260808 08:27:59.094882 17117 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.194@KRBTEST.COM'} at 127.22.235.194:38351
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab.
Entry for principal kudu/127.22.235.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab.
I20260808 08:27:59.111222 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.195:0
--local_ip_for_outbound_sockets=127.22.235.195
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:46859
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab
--principal=kudu/127.22.235.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:41847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:27:59.222020 17643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:27:59.222265 17643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:27:59.222325 17643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:27:59.226257 17643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:27:59.226325 17643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:27:59.226401 17643 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.195
I20260808 08:27:59.231503 17643 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:41847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab
--principal=kudu/127.22.235.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:46859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17643
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.195
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:27:59.232409 17643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:27:59.233392 17643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:27:59.242730 17655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:27:59.242774 17652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:27:59.242873 17643 server_base.cc:1034] running on GCE node
W20260808 08:27:59.242983 17653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:27:59.243399 17643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:27:59.244011 17643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:27:59.245172 17643 hybrid_clock.cc:648] HybridClock initialized: now 1786177679245157 us; error 25 us; skew 500 ppm
Aug 08 08:27:59 dist-test-slave-f7th krb5kdc[17045](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177679, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:27:59.248531 17643 init.cc:377] Logged in from keytab as kudu/127.22.235.195@KRBTEST.COM (short username kudu)
I20260808 08:27:59.249625 17643 webserver.cc:460] Webserver started at http://127.22.235.195:33897/ using document root <none> and password file <none>
I20260808 08:27:59.250396 17643 fs_manager.cc:359] Metadata directory not provided
I20260808 08:27:59.250490 17643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:27:59.250746 17643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:27:59.253124 17643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-2/data/instance:
uuid: "622af630070e4d129fdbd0e90b1cfd84"
format_stamp: "Formatted at 2026-08-08 08:27:59 on dist-test-slave-f7th"
server_key: "c5bd78cf439e821a2bed4fde1abf1581"
server_key_iv: "95f36dcf00e9bfd3e03d6bec3c21cb74"
server_key_version: "encryptionkey@0"
I20260808 08:27:59.253633 17643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance:
uuid: "622af630070e4d129fdbd0e90b1cfd84"
format_stamp: "Formatted at 2026-08-08 08:27:59 on dist-test-slave-f7th"
server_key: "c5bd78cf439e821a2bed4fde1abf1581"
server_key_iv: "95f36dcf00e9bfd3e03d6bec3c21cb74"
server_key_version: "encryptionkey@0"
I20260808 08:27:59.257365 17643 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 08:27:59.259781 17663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:59.261067 17643 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 08:27:59.261168 17643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-2/wal
uuid: "622af630070e4d129fdbd0e90b1cfd84"
format_stamp: "Formatted at 2026-08-08 08:27:59 on dist-test-slave-f7th"
server_key: "c5bd78cf439e821a2bed4fde1abf1581"
server_key_iv: "95f36dcf00e9bfd3e03d6bec3c21cb74"
server_key_version: "encryptionkey@0"
I20260808 08:27:59.261277 17643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:27:59.278226 17643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:27:59.280896 17643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:27:59.281069 17643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:27:59.281642 17643 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:27:59.282449 17643 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:27:59.282495 17643 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:59.282538 17643 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:27:59.282568 17643 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:27:59.291301 17643 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.195:34613
I20260808 08:27:59.291322 17863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.195:34613 every 8 connection(s)
I20260808 08:27:59.292160 17643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
I20260808 08:27:59.297645 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 17643
I20260808 08:27:59.297766 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance
I20260808 08:27:59.298072 23471 external_mini_cluster.cc:1428] Setting key ef9752e569b4a83001c765f430953fab
Aug 08 08:27:59 dist-test-slave-f7th krb5kdc[17045](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177679, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.195@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:27:59.306980 17181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:59.293921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:37729 (local address 127.22.235.254:46859)
0808 08:27:59.294049 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:59.294052 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:59.294938 (+ 886us) server_negotiation.cc:407] Connection header received
0808 08:27:59.296525 (+ 1587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:59.296529 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:59.296591 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:59.296695 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:27:59.298077 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:59.298864 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:59.300511 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:59.300842 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:59.300956 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:59.303723 (+ 2767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:27:59.303727 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:27:59.303729 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:27:59.303754 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:27:59.305193 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:59.305743 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:59.305745 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:59.305747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:59.305817 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:59.306086 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:59.306088 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:59.306089 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:59.306249 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:27:59.306312 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:59.306749 (+ 437us) server_negotiation.cc:299] Negotiation successful
0808 08:27:59.306856 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 08:27:59.307880 17867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:59.294348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.195:37729)
0808 08:27:59.294803 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:59.294850 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:59.296305 (+ 1455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:59.296863 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:27:59.296870 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:27:59.297227 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:27:59.297867 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:59.297889 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:59.299012 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:59.299017 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:59.300386 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:59.300398 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:59.301021 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:59.301026 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:59.301221 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:59.301834 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:27:59.301854 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:27:59.303519 (+ 1665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:27:59.305281 (+ 1762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:59.305286 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:59.305298 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:59.305656 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:59.305882 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:27:59.305884 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:27:59.305885 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:27:59.306024 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:27:59.306322 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:27:59.306326 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:27:59.306567 (+ 241us) client_negotiation.cc:769] Sending connection context
0808 08:27:59.306718 (+ 151us) client_negotiation.cc:240] Negotiation successful
0808 08:27:59.306981 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":110,"threads_started":1}
I20260808 08:27:59.309067 17864 heartbeater.cc:347] Connected to a master server at 127.22.235.254:46859
I20260808 08:27:59.309296 17864 heartbeater.cc:464] Registering TS with master...
I20260808 08:27:59.309769 17864 heartbeater.cc:511] Master 127.22.235.254:46859 requested a full tablet report, sending...
I20260808 08:27:59.310824 17117 ts_manager.cc:194] Registered new tserver with Master: 622af630070e4d129fdbd0e90b1cfd84 (127.22.235.195:34613)
I20260808 08:27:59.311538 17117 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.195@KRBTEST.COM'} at 127.22.235.195:37729
I20260808 08:27:59.312824 23471 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 08:27:59.322198 17181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:59.315025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51220 (local address 127.22.235.254:46859)
0808 08:27:59.315150 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:59.315154 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:59.315333 (+ 179us) server_negotiation.cc:407] Connection header received
0808 08:27:59.315455 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:59.315458 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:59.315517 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:59.315582 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:27:59.316289 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:59.316909 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:59.317727 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:59.317939 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:59.318027 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:59.319328 (+ 1301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:27:59.319333 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:27:59.319336 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:27:59.319361 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:27:59.320503 (+ 1142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:59.320933 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:59.320937 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:59.320939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:59.321018 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:27:59.321267 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:27:59.321271 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:27:59.321273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:27:59.321524 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:27:59.321648 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:59.321932 (+ 284us) server_negotiation.cc:299] Negotiation successful
0808 08:27:59.322046 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 08:27:59.325057 17117 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51220:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 08:27:59.328089 17117 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 08:27:59.341593 17881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:59.336098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:45483 (local address 127.0.0.1:56866)
0808 08:27:59.336541 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:59.336596 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:59.336760 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:59.337568 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:27:59.337574 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:27:59.337599 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:27:59.337722 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:59.337728 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:59.339029 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:59.339032 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:59.340162 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:59.340173 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:59.341058 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:59.341061 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:59.341170 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:59.341209 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 08:27:59.341323 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 08:27:59.341445 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":65,"threads_started":1}
I20260808 08:27:59.341629 17882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:59.336273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56866 (local address 127.22.235.193:45483)
0808 08:27:59.336577 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:59.336596 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:59.336699 (+ 103us) server_negotiation.cc:407] Connection header received
0808 08:27:59.336979 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:59.336986 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:59.337201 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:59.337358 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:27:59.337833 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:59.338920 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:59.340276 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:59.340944 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:59.341059 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:59.341203 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:59.341373 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 08:27:59.341479 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":109,"threads_started":1}
I20260808 08:27:59.342707 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:59.336825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.194:39403 (local address 127.0.0.1:36664)
0808 08:27:59.337942 (+ 1117us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:59.337955 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:59.338059 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:59.338419 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:27:59.338422 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:27:59.338433 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:27:59.338530 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:59.338536 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:59.339901 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:59.339904 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:59.341133 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:59.341145 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:59.342288 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:59.342291 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:59.342378 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:59.342400 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 08:27:59.342473 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 08:27:59.342570 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1054,"mutex_wait_us":3,"thread_start_us":104,"threads_started":1}
I20260808 08:27:59.342952 17884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:59.336983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36664 (local address 127.22.235.194:39403)
0808 08:27:59.337455 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:59.337466 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:59.338051 (+ 585us) server_negotiation.cc:407] Connection header received
0808 08:27:59.338150 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:59.338154 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:59.338291 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:59.338441 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:27:59.338609 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:59.339754 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:59.341274 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:59.341989 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:59.342104 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:59.342226 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:59.342534 (+ 308us) server_negotiation.cc:299] Negotiation successful
0808 08:27:59.342635 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":113,"threads_started":1}
I20260808 08:27:59.343087 17883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:59.337163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.195:34613 (local address 127.0.0.1:42572)
0808 08:27:59.338238 (+ 1075us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:59.338250 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:59.338329 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:59.338719 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:27:59.338721 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:27:59.338737 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:27:59.338803 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:59.338807 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:59.340068 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:59.340071 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:59.341195 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:59.341203 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:59.342335 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:59.342338 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:59.342402 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:59.342421 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 08:27:59.342492 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 08:27:59.342973 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1015,"mutex_wait_us":396,"thread_start_us":86,"threads_started":1}
I20260808 08:27:59.343564 17887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:59.337561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42572 (local address 127.22.235.195:34613)
0808 08:27:59.337887 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:59.337895 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:59.338338 (+ 443us) server_negotiation.cc:407] Connection header received
0808 08:27:59.338418 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:59.338423 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:59.338598 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:59.338745 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:27:59.338885 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:59.339923 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:59.341289 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:59.342068 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:59.342170 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:59.342278 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:59.342571 (+ 293us) server_negotiation.cc:299] Negotiation successful
0808 08:27:59.343414 (+ 843us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":148,"threads_started":1}
I20260808 08:27:59.344919 17316 tablet_service.cc:1467] Processing CreateTablet for tablet 6fd0bb5652e7460ca157bf38452f7182 (DEFAULT_TABLE table=test-table [id=f423175e7750450b84e6c10568a3ac71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:27:59.345889 17316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fd0bb5652e7460ca157bf38452f7182. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:27:59.345867 17522 tablet_service.cc:1467] Processing CreateTablet for tablet 6fd0bb5652e7460ca157bf38452f7182 (DEFAULT_TABLE table=test-table [id=f423175e7750450b84e6c10568a3ac71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:27:59.346107 17754 tablet_service.cc:1467] Processing CreateTablet for tablet 6fd0bb5652e7460ca157bf38452f7182 (DEFAULT_TABLE table=test-table [id=f423175e7750450b84e6c10568a3ac71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:27:59.346980 17522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fd0bb5652e7460ca157bf38452f7182. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:27:59.346997 17754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fd0bb5652e7460ca157bf38452f7182. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:27:59.352145 17888 tablet_bootstrap.cc:492] T 6fd0bb5652e7460ca157bf38452f7182 P 5e6519a7db6245f9b459ee5905af464b: Bootstrap starting.
I20260808 08:27:59.354506 17888 tablet_bootstrap.cc:654] T 6fd0bb5652e7460ca157bf38452f7182 P 5e6519a7db6245f9b459ee5905af464b: Neither blocks nor log segments found. Creating new log.
I20260808 08:27:59.354740 17889 tablet_bootstrap.cc:492] T 6fd0bb5652e7460ca157bf38452f7182 P 622af630070e4d129fdbd0e90b1cfd84: Bootstrap starting.
I20260808 08:27:59.355624 17888 log.cc:824] T 6fd0bb5652e7460ca157bf38452f7182 P 5e6519a7db6245f9b459ee5905af464b: Log is configured to *not* fsync() on all Append() calls
I20260808 08:27:59.356608 17889 tablet_bootstrap.cc:654] T 6fd0bb5652e7460ca157bf38452f7182 P 622af630070e4d129fdbd0e90b1cfd84: Neither blocks nor log segments found. Creating new log.
I20260808 08:27:59.357002 17890 tablet_bootstrap.cc:492] T 6fd0bb5652e7460ca157bf38452f7182 P 23f05ff8a1b04bb0a5a11717faec4639: Bootstrap starting.
I20260808 08:27:59.357631 17888 tablet_bootstrap.cc:492] T 6fd0bb5652e7460ca157bf38452f7182 P 5e6519a7db6245f9b459ee5905af464b: No bootstrap required, opened a new log
I20260808 08:27:59.357847 17888 ts_tablet_manager.cc:1397] T 6fd0bb5652e7460ca157bf38452f7182 P 5e6519a7db6245f9b459ee5905af464b: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 08:27:59.357836 17889 log.cc:824] T 6fd0bb5652e7460ca157bf38452f7182 P 622af630070e4d129fdbd0e90b1cfd84: Log is configured to *not* fsync() on all Append() calls
I20260808 08:27:59.359433 17890 tablet_bootstrap.cc:654] T 6fd0bb5652e7460ca157bf38452f7182 P 23f05ff8a1b04bb0a5a11717faec4639: Neither blocks nor log segments found. Creating new log.
I20260808 08:27:59.359921 17889 tablet_bootstrap.cc:492] T 6fd0bb5652e7460ca157bf38452f7182 P 622af630070e4d129fdbd0e90b1cfd84: No bootstrap required, opened a new log
I20260808 08:27:59.360116 17889 ts_tablet_manager.cc:1397] T 6fd0bb5652e7460ca157bf38452f7182 P 622af630070e4d129fdbd0e90b1cfd84: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 08:27:59.360244 17890 log.cc:824] T 6fd0bb5652e7460ca157bf38452f7182 P 23f05ff8a1b04bb0a5a11717faec4639: Log is configured to *not* fsync() on all Append() calls
I20260808 08:27:59.361415 17888 raft_consensus.cc:348] T 6fd0bb5652e7460ca157bf38452f7182 P 5e6519a7db6245f9b459ee5905af464b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6519a7db6245f9b459ee5905af464b" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45483 } } peers { permanent_uuid: "23f05ff8a1b04bb0a5a11717faec4639" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39403 } } peers { permanent_uuid: "622af630070e4d129fdbd0e90b1cfd84" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34613 } }
I20260808 08:27:59.361630 17888 raft_consensus.cc:374] T 6fd0bb5652e7460ca157bf38452f7182 P 5e6519a7db6245f9b459ee5905af464b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:27:59.361692 17888 raft_consensus.cc:729] T 6fd0bb5652e7460ca157bf38452f7182 P 5e6519a7db6245f9b459ee5905af464b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e6519a7db6245f9b459ee5905af464b, State: Initialized, Role: FOLLOWER
I20260808 08:27:59.362099 17888 consensus_queue.cc:260] T 6fd0bb5652e7460ca157bf38452f7182 P 5e6519a7db6245f9b459ee5905af464b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6519a7db6245f9b459ee5905af464b" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45483 } } peers { permanent_uuid: "23f05ff8a1b04bb0a5a11717faec4639" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39403 } } peers { permanent_uuid: "622af630070e4d129fdbd0e90b1cfd84" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34613 } }
I20260808 08:27:59.362216 17890 tablet_bootstrap.cc:492] T 6fd0bb5652e7460ca157bf38452f7182 P 23f05ff8a1b04bb0a5a11717faec4639: No bootstrap required, opened a new log
I20260808 08:27:59.362388 17890 ts_tablet_manager.cc:1397] T 6fd0bb5652e7460ca157bf38452f7182 P 23f05ff8a1b04bb0a5a11717faec4639: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 08:27:59.362728 17419 heartbeater.cc:503] Master 127.22.235.254:46859 was elected leader, sending a full tablet report...
I20260808 08:27:59.363771 17889 raft_consensus.cc:348] T 6fd0bb5652e7460ca157bf38452f7182 P 622af630070e4d129fdbd0e90b1cfd84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6519a7db6245f9b459ee5905af464b" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45483 } } peers { permanent_uuid: "23f05ff8a1b04bb0a5a11717faec4639" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39403 } } peers { permanent_uuid: "622af630070e4d129fdbd0e90b1cfd84" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34613 } }
I20260808 08:27:59.364053 17889 raft_consensus.cc:374] T 6fd0bb5652e7460ca157bf38452f7182 P 622af630070e4d129fdbd0e90b1cfd84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:27:59.364112 17889 raft_consensus.cc:729] T 6fd0bb5652e7460ca157bf38452f7182 P 622af630070e4d129fdbd0e90b1cfd84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 622af630070e4d129fdbd0e90b1cfd84, State: Initialized, Role: FOLLOWER
I20260808 08:27:59.364132 17888 ts_tablet_manager.cc:1428] T 6fd0bb5652e7460ca157bf38452f7182 P 5e6519a7db6245f9b459ee5905af464b: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 08:27:59.364579 17889 consensus_queue.cc:260] T 6fd0bb5652e7460ca157bf38452f7182 P 622af630070e4d129fdbd0e90b1cfd84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6519a7db6245f9b459ee5905af464b" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45483 } } peers { permanent_uuid: "23f05ff8a1b04bb0a5a11717faec4639" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39403 } } peers { permanent_uuid: "622af630070e4d129fdbd0e90b1cfd84" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34613 } }
I20260808 08:27:59.365294 17864 heartbeater.cc:503] Master 127.22.235.254:46859 was elected leader, sending a full tablet report...
I20260808 08:27:59.365670 17889 ts_tablet_manager.cc:1428] T 6fd0bb5652e7460ca157bf38452f7182 P 622af630070e4d129fdbd0e90b1cfd84: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
I20260808 08:27:59.366032 17890 raft_consensus.cc:348] T 6fd0bb5652e7460ca157bf38452f7182 P 23f05ff8a1b04bb0a5a11717faec4639 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6519a7db6245f9b459ee5905af464b" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45483 } } peers { permanent_uuid: "23f05ff8a1b04bb0a5a11717faec4639" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39403 } } peers { permanent_uuid: "622af630070e4d129fdbd0e90b1cfd84" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34613 } }
I20260808 08:27:59.366441 17890 raft_consensus.cc:374] T 6fd0bb5652e7460ca157bf38452f7182 P 23f05ff8a1b04bb0a5a11717faec4639 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:27:59.366521 17890 raft_consensus.cc:729] T 6fd0bb5652e7460ca157bf38452f7182 P 23f05ff8a1b04bb0a5a11717faec4639 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23f05ff8a1b04bb0a5a11717faec4639, State: Initialized, Role: FOLLOWER
I20260808 08:27:59.366942 17890 consensus_queue.cc:260] T 6fd0bb5652e7460ca157bf38452f7182 P 23f05ff8a1b04bb0a5a11717faec4639 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6519a7db6245f9b459ee5905af464b" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45483 } } peers { permanent_uuid: "23f05ff8a1b04bb0a5a11717faec4639" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39403 } } peers { permanent_uuid: "622af630070e4d129fdbd0e90b1cfd84" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34613 } }
I20260808 08:27:59.367585 17633 heartbeater.cc:503] Master 127.22.235.254:46859 was elected leader, sending a full tablet report...
I20260808 08:27:59.367779 17890 ts_tablet_manager.cc:1428] T 6fd0bb5652e7460ca157bf38452f7182 P 23f05ff8a1b04bb0a5a11717faec4639: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260808 08:27:59.543134 17865 tablet.cc:2369] T 6fd0bb5652e7460ca157bf38452f7182 P 622af630070e4d129fdbd0e90b1cfd84: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 08:27:59.577720 17634 tablet.cc:2369] T 6fd0bb5652e7460ca157bf38452f7182 P 23f05ff8a1b04bb0a5a11717faec4639: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 08:27:59.580844 17900 raft_consensus.cc:482] T 6fd0bb5652e7460ca157bf38452f7182 P 5e6519a7db6245f9b459ee5905af464b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 08:27:59.581056 17900 raft_consensus.cc:504] T 6fd0bb5652e7460ca157bf38452f7182 P 5e6519a7db6245f9b459ee5905af464b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6519a7db6245f9b459ee5905af464b" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45483 } } peers { permanent_uuid: "23f05ff8a1b04bb0a5a11717faec4639" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39403 } } peers { permanent_uuid: "622af630070e4d129fdbd0e90b1cfd84" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34613 } }
I20260808 08:27:59.582280 17900 leader_election.cc:288] T 6fd0bb5652e7460ca157bf38452f7182 P 5e6519a7db6245f9b459ee5905af464b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23f05ff8a1b04bb0a5a11717faec4639 (127.22.235.194:39403), 622af630070e4d129fdbd0e90b1cfd84 (127.22.235.195:34613)
I20260808 08:27:59.587172 17887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:59.583137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:55995 (local address 127.22.235.195:34613)
0808 08:27:59.583352 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:59.583356 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:59.583374 (+ 18us) server_negotiation.cc:407] Connection header received
0808 08:27:59.583542 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:59.583545 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:59.583595 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:59.583671 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:27:59.583898 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:59.584374 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:59.586038 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:59.586629 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:59.586746 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:59.586815 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:59.586998 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 08:27:59.587073 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126}
I20260808 08:27:59.587193 17904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:59.583006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.195:34613 (local address 127.22.235.193:55995)
0808 08:27:59.583307 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:59.583320 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:59.583436 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:59.583690 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:27:59.583693 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:27:59.583708 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:27:59.583805 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:59.583809 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:59.584473 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:59.584476 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:59.585904 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:59.585917 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:59.586749 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:59.586752 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:59.586860 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:59.586884 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 08:27:59.586956 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 08:27:59.587032 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":94,"spinlock_wait_cycles":896,"thread_start_us":118,"threads_started":1}
I20260808 08:27:59.588019 17884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:59.582617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:36181 (local address 127.22.235.194:39403)
0808 08:27:59.582762 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:59.582766 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:59.582908 (+ 142us) server_negotiation.cc:407] Connection header received
0808 08:27:59.583095 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:59.583100 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:59.583158 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:59.583211 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:27:59.583647 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:59.584428 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:59.586872 (+ 2444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:59.587489 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:59.587601 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:59.587646 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:59.587848 (+ 202us) server_negotiation.cc:299] Negotiation successful
0808 08:27:59.587921 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 08:27:59.588033 17903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:59.582513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.194:39403 (local address 127.22.235.193:36181)
0808 08:27:59.582812 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:59.582827 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 08:27:59.582950 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:27:59.583382 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:27:59.583385 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:27:59.583406 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:27:59.583524 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:59.583530 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:59.584517 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:59.584519 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:59.585904 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:27:59.585917 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:27:59.587656 (+ 1739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:27:59.587658 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:27:59.587733 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:59.587750 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 08:27:59.587821 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 08:27:59.587889 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":71,"spinlock_wait_cycles":7936,"thread_start_us":121,"threads_started":1}
I20260808 08:27:59.588150 17793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fd0bb5652e7460ca157bf38452f7182" candidate_uuid: "5e6519a7db6245f9b459ee5905af464b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "622af630070e4d129fdbd0e90b1cfd84" is_pre_election: true
I20260808 08:27:59.588430 17793 raft_consensus.cc:2454] T 6fd0bb5652e7460ca157bf38452f7182 P 622af630070e4d129fdbd0e90b1cfd84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e6519a7db6245f9b459ee5905af464b in term 0.
I20260808 08:27:59.588580 17553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fd0bb5652e7460ca157bf38452f7182" candidate_uuid: "5e6519a7db6245f9b459ee5905af464b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23f05ff8a1b04bb0a5a11717faec4639" is_pre_election: true
I20260808 08:27:59.588877 17553 raft_consensus.cc:2454] T 6fd0bb5652e7460ca157bf38452f7182 P 23f05ff8a1b04bb0a5a11717faec4639 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e6519a7db6245f9b459ee5905af464b in term 0.
I20260808 08:27:59.588912 17250 leader_election.cc:302] T 6fd0bb5652e7460ca157bf38452f7182 P 5e6519a7db6245f9b459ee5905af464b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e6519a7db6245f9b459ee5905af464b, 622af630070e4d129fdbd0e90b1cfd84; no voters:
I20260808 08:27:59.589192 17900 raft_consensus.cc:2781] T 6fd0bb5652e7460ca157bf38452f7182 P 5e6519a7db6245f9b459ee5905af464b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 08:27:59.589277 17900 raft_consensus.cc:482] T 6fd0bb5652e7460ca157bf38452f7182 P 5e6519a7db6245f9b459ee5905af464b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 08:27:59.589327 17900 raft_consensus.cc:3037] T 6fd0bb5652e7460ca157bf38452f7182 P 5e6519a7db6245f9b459ee5905af464b [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:27:59.590338 17900 raft_consensus.cc:504] T 6fd0bb5652e7460ca157bf38452f7182 P 5e6519a7db6245f9b459ee5905af464b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6519a7db6245f9b459ee5905af464b" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45483 } } peers { permanent_uuid: "23f05ff8a1b04bb0a5a11717faec4639" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39403 } } peers { permanent_uuid: "622af630070e4d129fdbd0e90b1cfd84" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34613 } }
I20260808 08:27:59.590662 17900 leader_election.cc:288] T 6fd0bb5652e7460ca157bf38452f7182 P 5e6519a7db6245f9b459ee5905af464b [CANDIDATE]: Term 1 election: Requested vote from peers 23f05ff8a1b04bb0a5a11717faec4639 (127.22.235.194:39403), 622af630070e4d129fdbd0e90b1cfd84 (127.22.235.195:34613)
I20260808 08:27:59.591015 17553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fd0bb5652e7460ca157bf38452f7182" candidate_uuid: "5e6519a7db6245f9b459ee5905af464b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23f05ff8a1b04bb0a5a11717faec4639"
I20260808 08:27:59.591019 17793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fd0bb5652e7460ca157bf38452f7182" candidate_uuid: "5e6519a7db6245f9b459ee5905af464b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "622af630070e4d129fdbd0e90b1cfd84"
I20260808 08:27:59.591125 17553 raft_consensus.cc:3037] T 6fd0bb5652e7460ca157bf38452f7182 P 23f05ff8a1b04bb0a5a11717faec4639 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:27:59.591135 17793 raft_consensus.cc:3037] T 6fd0bb5652e7460ca157bf38452f7182 P 622af630070e4d129fdbd0e90b1cfd84 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:27:59.592262 17793 raft_consensus.cc:2454] T 6fd0bb5652e7460ca157bf38452f7182 P 622af630070e4d129fdbd0e90b1cfd84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e6519a7db6245f9b459ee5905af464b in term 1.
I20260808 08:27:59.592551 17553 raft_consensus.cc:2454] T 6fd0bb5652e7460ca157bf38452f7182 P 23f05ff8a1b04bb0a5a11717faec4639 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e6519a7db6245f9b459ee5905af464b in term 1.
I20260808 08:27:59.592605 17250 leader_election.cc:302] T 6fd0bb5652e7460ca157bf38452f7182 P 5e6519a7db6245f9b459ee5905af464b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e6519a7db6245f9b459ee5905af464b, 622af630070e4d129fdbd0e90b1cfd84; no voters:
I20260808 08:27:59.592818 17900 raft_consensus.cc:2781] T 6fd0bb5652e7460ca157bf38452f7182 P 5e6519a7db6245f9b459ee5905af464b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:27:59.593045 17900 raft_consensus.cc:686] T 6fd0bb5652e7460ca157bf38452f7182 P 5e6519a7db6245f9b459ee5905af464b [term 1 LEADER]: Becoming Leader. State: Replica: 5e6519a7db6245f9b459ee5905af464b, State: Running, Role: LEADER
I20260808 08:27:59.593370 17900 consensus_queue.cc:237] T 6fd0bb5652e7460ca157bf38452f7182 P 5e6519a7db6245f9b459ee5905af464b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6519a7db6245f9b459ee5905af464b" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45483 } } peers { permanent_uuid: "23f05ff8a1b04bb0a5a11717faec4639" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39403 } } peers { permanent_uuid: "622af630070e4d129fdbd0e90b1cfd84" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34613 } }
I20260808 08:27:59.596566 17116 catalog_manager.cc:5495] T 6fd0bb5652e7460ca157bf38452f7182 P 5e6519a7db6245f9b459ee5905af464b reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e6519a7db6245f9b459ee5905af464b (127.22.235.193). New cstate: current_term: 1 leader_uuid: "5e6519a7db6245f9b459ee5905af464b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6519a7db6245f9b459ee5905af464b" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "23f05ff8a1b04bb0a5a11717faec4639" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "622af630070e4d129fdbd0e90b1cfd84" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34613 } health_report { overall_health: UNKNOWN } } }
W20260808 08:27:59.599684 17420 tablet.cc:2369] T 6fd0bb5652e7460ca157bf38452f7182 P 5e6519a7db6245f9b459ee5905af464b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 08:27:59.634246 17882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:59.630836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56882 (local address 127.22.235.193:45483)
0808 08:27:59.630960 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 08:27:59.630963 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:27:59.630977 (+ 14us) server_negotiation.cc:407] Connection header received
0808 08:27:59.631116 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:27:59.631118 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:27:59.631179 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:27:59.631273 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:27:59.631514 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:59.632014 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:59.632933 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:27:59.633134 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:27:59.633199 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:27:59.633428 (+ 229us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:27:59.633877 (+ 449us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 08:27:59.633968 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 08:27:59.634083 (+ 115us) server_negotiation.cc:299] Negotiation successful
0808 08:27:59.634133 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 08:27:59.640113 17793 raft_consensus.cc:1261] T 6fd0bb5652e7460ca157bf38452f7182 P 622af630070e4d129fdbd0e90b1cfd84 [term 1 FOLLOWER]: Refusing update from remote peer 5e6519a7db6245f9b459ee5905af464b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:27:59.640113 17553 raft_consensus.cc:1261] T 6fd0bb5652e7460ca157bf38452f7182 P 23f05ff8a1b04bb0a5a11717faec4639 [term 1 FOLLOWER]: Refusing update from remote peer 5e6519a7db6245f9b459ee5905af464b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:27:59.640810 17900 consensus_queue.cc:1035] T 6fd0bb5652e7460ca157bf38452f7182 P 5e6519a7db6245f9b459ee5905af464b [LEADER]: Connected to new peer: Peer: permanent_uuid: "622af630070e4d129fdbd0e90b1cfd84" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:27:59.641026 17906 consensus_queue.cc:1035] T 6fd0bb5652e7460ca157bf38452f7182 P 5e6519a7db6245f9b459ee5905af464b [LEADER]: Connected to new peer: Peer: permanent_uuid: "23f05ff8a1b04bb0a5a11717faec4639" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:27:59.651074 17912 mvcc.cc:205] Tried to move back new op lower bound from 7316183775798603776 to 7316183775616860160. Current Snapshot: MvccSnapshot[applied={T|T < 7316183775798603776}]
I20260808 08:27:59.655768 17117 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:27:59.655947 17117 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:27:59.658349 17117 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676: Sending DeleteTablet for 3 replicas of tablet 6fd0bb5652e7460ca157bf38452f7182
I20260808 08:27:59.659289 17316 tablet_service.cc:1514] Processing DeleteTablet for tablet 6fd0bb5652e7460ca157bf38452f7182 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:27:59 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:56866
I20260808 08:27:59.659471 17754 tablet_service.cc:1514] Processing DeleteTablet for tablet 6fd0bb5652e7460ca157bf38452f7182 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:27:59 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:42572
I20260808 08:27:59.659473 17522 tablet_service.cc:1514] Processing DeleteTablet for tablet 6fd0bb5652e7460ca157bf38452f7182 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:27:59 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:36664
I20260808 08:27:59.659809 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 17222
I20260808 08:27:59.659929 17928 tablet_replica.cc:330] T 6fd0bb5652e7460ca157bf38452f7182 P 23f05ff8a1b04bb0a5a11717faec4639: stopping tablet replica
I20260808 08:27:59.668965 17927 tablet_replica.cc:330] T 6fd0bb5652e7460ca157bf38452f7182 P 622af630070e4d129fdbd0e90b1cfd84: stopping tablet replica
I20260808 08:27:59.669236 17928 raft_consensus.cc:2229] T 6fd0bb5652e7460ca157bf38452f7182 P 23f05ff8a1b04bb0a5a11717faec4639 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 08:27:59.669323 17927 raft_consensus.cc:2229] T 6fd0bb5652e7460ca157bf38452f7182 P 622af630070e4d129fdbd0e90b1cfd84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 08:27:59.669670 17927 raft_consensus.cc:2258] T 6fd0bb5652e7460ca157bf38452f7182 P 622af630070e4d129fdbd0e90b1cfd84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:27:59.669700 17928 raft_consensus.cc:2258] T 6fd0bb5652e7460ca157bf38452f7182 P 23f05ff8a1b04bb0a5a11717faec4639 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:27:59.671284 17927 ts_tablet_manager.cc:1905] T 6fd0bb5652e7460ca157bf38452f7182 P 622af630070e4d129fdbd0e90b1cfd84: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:27:59.672577 17928 ts_tablet_manager.cc:1905] T 6fd0bb5652e7460ca157bf38452f7182 P 23f05ff8a1b04bb0a5a11717faec4639: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:27:59.674091 17927 ts_tablet_manager.cc:1918] T 6fd0bb5652e7460ca157bf38452f7182 P 622af630070e4d129fdbd0e90b1cfd84: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 08:27:59.674253 17927 log.cc:1196] T 6fd0bb5652e7460ca157bf38452f7182 P 622af630070e4d129fdbd0e90b1cfd84: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-2/wal/wals/6fd0bb5652e7460ca157bf38452f7182
I20260808 08:27:59.674588 17927 ts_tablet_manager.cc:1939] T 6fd0bb5652e7460ca157bf38452f7182 P 622af630070e4d129fdbd0e90b1cfd84: Deleting consensus metadata
I20260808 08:27:59.675964 17928 ts_tablet_manager.cc:1918] T 6fd0bb5652e7460ca157bf38452f7182 P 23f05ff8a1b04bb0a5a11717faec4639: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 08:27:59.676100 17928 log.cc:1196] T 6fd0bb5652e7460ca157bf38452f7182 P 23f05ff8a1b04bb0a5a11717faec4639: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-1/wal/wals/6fd0bb5652e7460ca157bf38452f7182
I20260808 08:27:59.676478 17928 ts_tablet_manager.cc:1939] T 6fd0bb5652e7460ca157bf38452f7182 P 23f05ff8a1b04bb0a5a11717faec4639: Deleting consensus metadata
I20260808 08:27:59.676487 17086 catalog_manager.cc:4841] TS 622af630070e4d129fdbd0e90b1cfd84 (127.22.235.195:34613): tablet 6fd0bb5652e7460ca157bf38452f7182 (table test-table [id=f423175e7750450b84e6c10568a3ac71]) successfully deleted
I20260808 08:27:59.677682 17086 catalog_manager.cc:4841] TS 23f05ff8a1b04bb0a5a11717faec4639 (127.22.235.194:39403): tablet 6fd0bb5652e7460ca157bf38452f7182 (table test-table [id=f423175e7750450b84e6c10568a3ac71]) successfully deleted
W20260808 08:27:59.682966 17086 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.235.193:45483 (error 108)
I20260808 08:27:59.683233 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 17427
I20260808 08:27:59.684562 17883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:27:59.684152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:45483 (local address 127.0.0.1:56888)
0808 08:27:59.684377 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0808 08:27:59.684463 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.193:45483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
W20260808 08:27:59.684784 17086 catalog_manager.cc:4584] TS 5e6519a7db6245f9b459ee5905af464b (127.22.235.193:45483): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6fd0bb5652e7460ca157bf38452f7182: Network error: Client connection negotiation failed: client connection to 127.22.235.193:45483: connect: Connection refused (error 111)
I20260808 08:27:59.699553 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 17643
I20260808 08:27:59.709776 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 17063
I20260808 08:28:19.725462 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:46859
--webserver_interface=127.22.235.254
--webserver_port=34107
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:41847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:46859
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:28:19.838728 17930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:28:19.838982 17930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:28:19.839027 17930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:28:19.842934 17930 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 08:28:19.842991 17930 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 08:28:19.843014 17930 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 08:28:19.843026 17930 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:28:19.843043 17930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:28:19.843065 17930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:28:19.843096 17930 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:28:19.843122 17930 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:28:19.848376 17930 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:41847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:46859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:46859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.235.254
--webserver_port=34107
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17930
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:28:19.849455 17930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:28:19.850463 17930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:28:19.864202 17940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:28:19.864188 17941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:28:19.864203 17944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:28:19.864367 17930 server_base.cc:1034] running on GCE node
I20260808 08:28:19.865095 17930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:28:19.866225 17930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:28:19.867379 17930 hybrid_clock.cc:648] HybridClock initialized: now 1786177699867359 us; error 30 us; skew 500 ppm
Aug 08 08:28:19 dist-test-slave-f7th krb5kdc[17045](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177699, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:28:19.870071 17930 init.cc:377] Logged in from keytab as kudu/127.22.235.254@KRBTEST.COM (short username kudu)
I20260808 08:28:19.870934 17930 webserver.cc:460] Webserver started at http://127.22.235.254:34107/ using document root <none> and password file <none>
I20260808 08:28:19.871484 17930 fs_manager.cc:359] Metadata directory not provided
I20260808 08:28:19.871577 17930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:28:19.875337 17930 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:28:19.877419 17954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:28:19.878597 17930 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 08:28:19.878712 17930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "955105d3eda549ddb05259d3c312c676"
format_stamp: "Formatted at 2026-08-08 08:27:58 on dist-test-slave-f7th"
server_key: "14611b503cf030b60a6e621bfa39d2b0"
server_key_iv: "b669d78e0ca36526072a5faa4123b6f3"
server_key_version: "encryptionkey@0"
I20260808 08:28:19.879261 17930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:28:19.909904 17930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:28:19.912873 17930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:28:19.913051 17930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:28:19.919927 17930 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.254:46859
I20260808 08:28:19.920192 18036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.254:46859 every 8 connection(s)
I20260808 08:28:19.921032 17930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:28:19.922926 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 17930
I20260808 08:28:19.923377 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.193:45483
--local_ip_for_outbound_sockets=127.22.235.193
--tserver_master_addrs=127.22.235.254:46859
--webserver_port=32955
--webserver_interface=127.22.235.193
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab
--principal=kudu/127.22.235.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:41847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 08:28:19.926167 18037 sys_catalog.cc:263] Verifying existing consensus state
I20260808 08:28:19.927848 18037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676: Bootstrap starting.
I20260808 08:28:19.944803 18037 log.cc:824] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676: Log is configured to *not* fsync() on all Append() calls
I20260808 08:28:19.953562 18037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 08:28:19.953939 18037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676: Bootstrap complete.
I20260808 08:28:19.957816 18037 raft_consensus.cc:348] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "955105d3eda549ddb05259d3c312c676" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46859 } }
I20260808 08:28:19.958218 18037 raft_consensus.cc:729] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955105d3eda549ddb05259d3c312c676, State: Initialized, Role: FOLLOWER
I20260808 08:28:19.958688 18037 consensus_queue.cc:260] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "955105d3eda549ddb05259d3c312c676" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46859 } }
I20260808 08:28:19.958804 18037 raft_consensus.cc:388] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:28:19.958869 18037 raft_consensus.cc:482] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:28:19.958963 18037 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [term 1 FOLLOWER]: Advancing to term 2
I20260808 08:28:19.960871 18037 raft_consensus.cc:504] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "955105d3eda549ddb05259d3c312c676" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46859 } }
I20260808 08:28:19.961207 18037 leader_election.cc:302] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 955105d3eda549ddb05259d3c312c676; no voters:
I20260808 08:28:19.961514 18037 leader_election.cc:288] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 08:28:19.961615 18043 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 08:28:19.961820 18043 raft_consensus.cc:686] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [term 2 LEADER]: Becoming Leader. State: Replica: 955105d3eda549ddb05259d3c312c676, State: Running, Role: LEADER
I20260808 08:28:19.962178 18043 consensus_queue.cc:237] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "955105d3eda549ddb05259d3c312c676" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46859 } }
I20260808 08:28:19.962805 18037 sys_catalog.cc:564] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:28:19.963687 18045 sys_catalog.cc:455] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "955105d3eda549ddb05259d3c312c676" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "955105d3eda549ddb05259d3c312c676" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46859 } } }
I20260808 08:28:19.963817 18045 sys_catalog.cc:458] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [sys.catalog]: This master's current role is: LEADER
I20260808 08:28:19.964103 18047 sys_catalog.cc:455] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 955105d3eda549ddb05259d3c312c676. Latest consensus state: current_term: 2 leader_uuid: "955105d3eda549ddb05259d3c312c676" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "955105d3eda549ddb05259d3c312c676" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46859 } } }
I20260808 08:28:19.964174 18047 sys_catalog.cc:458] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [sys.catalog]: This master's current role is: LEADER
I20260808 08:28:19.964185 18056 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:28:19.969293 18056 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:28:19.970245 18056 catalog_manager.cc:1259] Loaded cluster ID: 01abe1afe9464a23bca90e7f4fed11c1
I20260808 08:28:19.970314 18056 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:28:19.972877 18056 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:28:19.973606 18056 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676: Loaded TSK: 0
I20260808 08:28:19.989557 18056 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676: Generated new TSK 1
I20260808 08:28:19.989809 18056 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 08:28:20.040095 18039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:28:20.040349 18039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:28:20.040412 18039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:28:20.044335 18039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:28:20.044410 18039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:28:20.044492 18039 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.193
I20260808 08:28:20.049890 18039 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:41847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab
--principal=kudu/127.22.235.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.193:45483
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.235.193
--webserver_port=32955
--tserver_master_addrs=127.22.235.254:46859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.193
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:28:20.050956 18039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:28:20.051985 18039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:28:20.062762 18081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:28:20.062743 18080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:28:20.062716 18085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:28:20.063196 18039 server_base.cc:1034] running on GCE node
I20260808 08:28:20.063616 18039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:28:20.064323 18039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:28:20.065482 18039 hybrid_clock.cc:648] HybridClock initialized: now 1786177700065440 us; error 58 us; skew 500 ppm
Aug 08 08:28:20 dist-test-slave-f7th krb5kdc[17045](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177700, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:28:20.068379 18039 init.cc:377] Logged in from keytab as kudu/127.22.235.193@KRBTEST.COM (short username kudu)
I20260808 08:28:20.069530 18039 webserver.cc:460] Webserver started at http://127.22.235.193:32955/ using document root <none> and password file <none>
I20260808 08:28:20.070240 18039 fs_manager.cc:359] Metadata directory not provided
I20260808 08:28:20.070351 18039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:28:20.075358 18039 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260808 08:28:20.078157 18096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:28:20.079716 18039 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 08:28:20.079851 18039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-0/wal
uuid: "5e6519a7db6245f9b459ee5905af464b"
format_stamp: "Formatted at 2026-08-08 08:27:58 on dist-test-slave-f7th"
server_key: "19f824c19d97b739b738edbccf4247cd"
server_key_iv: "7d3c06265e9b909dba1c2f01ae18a614"
server_key_version: "encryptionkey@0"
I20260808 08:28:20.080571 18039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:28:20.124971 18039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:28:20.128173 18039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:28:20.128329 18039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:28:20.128925 18039 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:28:20.129954 18105 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 08:28:20.134197 18039 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 08:28:20.134253 18039 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260808 08:28:20.134311 18039 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 08:28:20.135952 18039 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 08:28:20.135993 18039 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260808 08:28:20.136301 18105 tablet_bootstrap.cc:492] T 6fd0bb5652e7460ca157bf38452f7182 P 5e6519a7db6245f9b459ee5905af464b: Bootstrap starting.
I20260808 08:28:20.145556 18039 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.193:45483
I20260808 08:28:20.145617 18281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.193:45483 every 8 connection(s)
I20260808 08:28:20.146472 18039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
I20260808 08:28:20.151464 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 18039
I20260808 08:28:20.151978 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.194:39403
--local_ip_for_outbound_sockets=127.22.235.194
--tserver_master_addrs=127.22.235.254:46859
--webserver_port=43027
--webserver_interface=127.22.235.194
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab
--principal=kudu/127.22.235.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:41847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 08:28:20.155113 18105 log.cc:824] T 6fd0bb5652e7460ca157bf38452f7182 P 5e6519a7db6245f9b459ee5905af464b: Log is configured to *not* fsync() on all Append() calls
Aug 08 08:28:20 dist-test-slave-f7th krb5kdc[17045](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177700, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:28:20.163429 18105 tablet_bootstrap.cc:492] T 6fd0bb5652e7460ca157bf38452f7182 P 5e6519a7db6245f9b459ee5905af464b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 08:28:20.163787 18105 tablet_bootstrap.cc:492] T 6fd0bb5652e7460ca157bf38452f7182 P 5e6519a7db6245f9b459ee5905af464b: Bootstrap complete.
I20260808 08:28:20.165055 18105 ts_tablet_manager.cc:1397] T 6fd0bb5652e7460ca157bf38452f7182 P 5e6519a7db6245f9b459ee5905af464b: Time spent bootstrapping tablet: real 0.029s user 0.006s sys 0.022s
I20260808 08:28:20.165504 18286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:20.148790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:45411 (local address 127.22.235.254:46859)
0808 08:28:20.149290 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0808 08:28:20.149310 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 08:28:20.149441 (+ 131us) server_negotiation.cc:407] Connection header received
0808 08:28:20.150819 (+ 1378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:28:20.150830 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:28:20.151131 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:28:20.151434 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:28:20.152760 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:20.153735 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:20.155410 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:20.156109 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:20.156275 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:28:20.159314 (+ 3039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:28:20.159321 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:28:20.159336 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:28:20.159372 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:28:20.161484 (+ 2112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:28:20.162164 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:28:20.162171 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:28:20.162176 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:28:20.162326 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:28:20.162730 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:28:20.162733 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:28:20.162735 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:28:20.163318 (+ 583us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:28:20.163450 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 08:28:20.164172 (+ 722us) server_negotiation.cc:299] Negotiation successful
0808 08:28:20.164465 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":128,"threads_started":1}
I20260808 08:28:20.166129 18285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:20.148679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.193:45411)
0808 08:28:20.149260 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:20.149319 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 08:28:20.150417 (+ 1098us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:28:20.151524 (+ 1107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:28:20.151533 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:28:20.151968 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:28:20.152559 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:28:20.152577 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:28:20.153877 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:28:20.153881 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:28:20.155271 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:28:20.155284 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:28:20.156258 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:28:20.156262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:28:20.156451 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:28:20.157127 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:28:20.157152 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:28:20.159084 (+ 1932us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:28:20.161626 (+ 2542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:28:20.161633 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:28:20.161648 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:28:20.162025 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:28:20.162427 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:28:20.162430 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:28:20.162432 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:28:20.162617 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:28:20.163445 (+ 828us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:28:20.163452 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:28:20.163769 (+ 317us) client_negotiation.cc:769] Sending connection context
0808 08:28:20.163990 (+ 221us) client_negotiation.cc:240] Negotiation successful
0808 08:28:20.165278 (+ 1288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":143,"threads_started":1}
I20260808 08:28:20.167865 18105 raft_consensus.cc:348] T 6fd0bb5652e7460ca157bf38452f7182 P 5e6519a7db6245f9b459ee5905af464b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6519a7db6245f9b459ee5905af464b" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45483 } } peers { permanent_uuid: "23f05ff8a1b04bb0a5a11717faec4639" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39403 } } peers { permanent_uuid: "622af630070e4d129fdbd0e90b1cfd84" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34613 } }
I20260808 08:28:20.168288 18105 raft_consensus.cc:729] T 6fd0bb5652e7460ca157bf38452f7182 P 5e6519a7db6245f9b459ee5905af464b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e6519a7db6245f9b459ee5905af464b, State: Initialized, Role: FOLLOWER
I20260808 08:28:20.168730 18282 heartbeater.cc:347] Connected to a master server at 127.22.235.254:46859
I20260808 08:28:20.168778 18105 consensus_queue.cc:260] T 6fd0bb5652e7460ca157bf38452f7182 P 5e6519a7db6245f9b459ee5905af464b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e6519a7db6245f9b459ee5905af464b" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45483 } } peers { permanent_uuid: "23f05ff8a1b04bb0a5a11717faec4639" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39403 } } peers { permanent_uuid: "622af630070e4d129fdbd0e90b1cfd84" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34613 } }
I20260808 08:28:20.168967 18282 heartbeater.cc:464] Registering TS with master...
I20260808 08:28:20.169528 18282 heartbeater.cc:511] Master 127.22.235.254:46859 requested a full tablet report, sending...
I20260808 08:28:20.169776 18105 ts_tablet_manager.cc:1428] T 6fd0bb5652e7460ca157bf38452f7182 P 5e6519a7db6245f9b459ee5905af464b: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260808 08:28:20.172124 17985 ts_manager.cc:194] Registered new tserver with Master: 5e6519a7db6245f9b459ee5905af464b (127.22.235.193:45483)
I20260808 08:28:20.176083 17985 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.193@KRBTEST.COM'} at 127.22.235.193:45411
I20260808 08:28:20.178895 18282 heartbeater.cc:503] Master 127.22.235.254:46859 was elected leader, sending a full tablet report...
Aug 08 08:28:20 dist-test-slave-f7th krb5kdc[17045](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177699, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for kudu/127.22.235.193@KRBTEST.COM
I20260808 08:28:20.185796 18294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:20.175510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37148 (local address 127.22.235.193:45483)
0808 08:28:20.176147 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0808 08:28:20.176159 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 08:28:20.176178 (+ 19us) server_negotiation.cc:407] Connection header received
0808 08:28:20.176246 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:28:20.176251 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:28:20.176407 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:28:20.176533 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:28:20.177223 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:20.178231 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:20.179394 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:20.179775 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:20.179870 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:28:20.182371 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:28:20.182379 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:28:20.182389 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:28:20.182411 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:28:20.183912 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:28:20.184414 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:28:20.184418 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:28:20.184419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:28:20.184491 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:28:20.184778 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:28:20.184781 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:28:20.184782 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:28:20.185026 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:28:20.185128 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 08:28:20.185509 (+ 381us) server_negotiation.cc:299] Negotiation successful
0808 08:28:20.185667 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":116,"threads_started":1}
I20260808 08:28:20.185796 18292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:20.175413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:45483 (local address 127.0.0.1:37148)
0808 08:28:20.175801 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:20.175843 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 08:28:20.175964 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:28:20.176549 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:28:20.176555 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:28:20.176967 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:28:20.177080 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:28:20.177087 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:28:20.178357 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:28:20.178361 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:28:20.179272 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:28:20.179285 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:28:20.179884 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:28:20.179888 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:28:20.179995 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:28:20.180525 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:28:20.180540 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:28:20.182244 (+ 1704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:28:20.184019 (+ 1775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:28:20.184028 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:28:20.184034 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:28:20.184319 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:28:20.184569 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:28:20.184572 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:28:20.184573 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:28:20.184694 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:28:20.185132 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:28:20.185138 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:28:20.185290 (+ 152us) client_negotiation.cc:769] Sending connection context
0808 08:28:20.185427 (+ 137us) client_negotiation.cc:240] Negotiation successful
0808 08:28:20.185629 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":119,"threads_started":1}
I20260808 08:28:20.187057 18167 tablet_service.cc:1514] Processing DeleteTablet for tablet 6fd0bb5652e7460ca157bf38452f7182 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:27:59 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:37148
I20260808 08:28:20.188005 18295 tablet_replica.cc:330] stopping tablet replica
I20260808 08:28:20.188282 18295 raft_consensus.cc:2229] T 6fd0bb5652e7460ca157bf38452f7182 P 5e6519a7db6245f9b459ee5905af464b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 08:28:20.188519 18295 raft_consensus.cc:2258] T 6fd0bb5652e7460ca157bf38452f7182 P 5e6519a7db6245f9b459ee5905af464b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:28:20.189435 18295 ts_tablet_manager.cc:1905] T 6fd0bb5652e7460ca157bf38452f7182 P 5e6519a7db6245f9b459ee5905af464b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:28:20.193163 18295 ts_tablet_manager.cc:1918] T 6fd0bb5652e7460ca157bf38452f7182 P 5e6519a7db6245f9b459ee5905af464b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 08:28:20.193243 18295 log.cc:1196] T 6fd0bb5652e7460ca157bf38452f7182 P 5e6519a7db6245f9b459ee5905af464b: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-0/wal/wals/6fd0bb5652e7460ca157bf38452f7182
I20260808 08:28:20.193475 18295 ts_tablet_manager.cc:1939] T 6fd0bb5652e7460ca157bf38452f7182 P 5e6519a7db6245f9b459ee5905af464b: Deleting consensus metadata
I20260808 08:28:20.195262 17955 catalog_manager.cc:4841] TS 5e6519a7db6245f9b459ee5905af464b (127.22.235.193:45483): tablet 6fd0bb5652e7460ca157bf38452f7182 (table test-table [id=f423175e7750450b84e6c10568a3ac71]) successfully deleted
W20260808 08:28:20.275820 18288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:28:20.276053 18288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:28:20.276111 18288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:28:20.280017 18288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:28:20.280067 18288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:28:20.280139 18288 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.194
I20260808 08:28:20.285207 18288 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:41847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab
--principal=kudu/127.22.235.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.194:39403
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.235.194
--webserver_port=43027
--tserver_master_addrs=127.22.235.254:46859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.194
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:28:20.286173 18288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:28:20.287220 18288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:28:20.296336 18306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:28:20.296370 18305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:28:20.296550 18288 server_base.cc:1034] running on GCE node
W20260808 08:28:20.296404 18308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:28:20.297073 18288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:28:20.297675 18288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:28:20.298830 18288 hybrid_clock.cc:648] HybridClock initialized: now 1786177700298815 us; error 27 us; skew 500 ppm
Aug 08 08:28:20 dist-test-slave-f7th krb5kdc[17045](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177700, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:28:20.305025 18288 init.cc:377] Logged in from keytab as kudu/127.22.235.194@KRBTEST.COM (short username kudu)
I20260808 08:28:20.305899 18288 webserver.cc:460] Webserver started at http://127.22.235.194:43027/ using document root <none> and password file <none>
I20260808 08:28:20.306432 18288 fs_manager.cc:359] Metadata directory not provided
I20260808 08:28:20.306505 18288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:28:20.310272 18288 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 08:28:20.312253 18319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:28:20.313555 18288 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:28:20.313648 18288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-1/wal
uuid: "23f05ff8a1b04bb0a5a11717faec4639"
format_stamp: "Formatted at 2026-08-08 08:27:59 on dist-test-slave-f7th"
server_key: "d5d81801df520fb0567eb4182667b79a"
server_key_iv: "a19e140047c133a8959da581b505119c"
server_key_version: "encryptionkey@0"
I20260808 08:28:20.314141 18288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:28:20.323994 18288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:28:20.327118 18288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:28:20.327265 18288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:28:20.327775 18288 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:28:20.328593 18288 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:28:20.328665 18288 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:28:20.328724 18288 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:28:20.328747 18288 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:28:20.337342 18288 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.194:39403
I20260808 08:28:20.337384 18495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.194:39403 every 8 connection(s)
I20260808 08:28:20.338340 18288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
I20260808 08:28:20.340415 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 18288
I20260808 08:28:20.340896 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.195:34613
--local_ip_for_outbound_sockets=127.22.235.195
--tserver_master_addrs=127.22.235.254:46859
--webserver_port=33897
--webserver_interface=127.22.235.195
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab
--principal=kudu/127.22.235.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:41847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 08:28:20 dist-test-slave-f7th krb5kdc[17045](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177700, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.194@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:28:20.352571 18286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:20.340046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:54987 (local address 127.22.235.254:46859)
0808 08:28:20.340183 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 08:28:20.340188 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:28:20.341077 (+ 889us) server_negotiation.cc:407] Connection header received
0808 08:28:20.342738 (+ 1661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:28:20.342742 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:28:20.342808 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:28:20.342903 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:28:20.344068 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:20.344663 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:20.346286 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:20.346519 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:20.346593 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:28:20.349275 (+ 2682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:28:20.349278 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:28:20.349280 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:28:20.349306 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:28:20.350623 (+ 1317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:28:20.351112 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:28:20.351115 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:28:20.351116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:28:20.351195 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:28:20.351509 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:28:20.351513 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:28:20.351514 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:28:20.351692 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:28:20.351779 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 08:28:20.352255 (+ 476us) server_negotiation.cc:299] Negotiation successful
0808 08:28:20.352395 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 08:28:20.353298 18498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:20.340389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.194:54987)
0808 08:28:20.340908 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:20.340965 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 08:28:20.342480 (+ 1515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:28:20.343096 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:28:20.343103 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:28:20.343441 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:28:20.343907 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:28:20.343920 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:28:20.344815 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:28:20.344820 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:28:20.346147 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:28:20.346161 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:28:20.346664 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:28:20.346669 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:28:20.346889 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:28:20.347454 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:28:20.347477 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:28:20.349108 (+ 1631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:28:20.350734 (+ 1626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:28:20.350740 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:28:20.350754 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:28:20.351014 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:28:20.351279 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:28:20.351282 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:28:20.351284 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:28:20.351415 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:28:20.351793 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:28:20.351800 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:28:20.352032 (+ 232us) client_negotiation.cc:769] Sending connection context
0808 08:28:20.352194 (+ 162us) client_negotiation.cc:240] Negotiation successful
0808 08:28:20.352503 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":149,"threads_started":1}
I20260808 08:28:20.354384 18496 heartbeater.cc:347] Connected to a master server at 127.22.235.254:46859
I20260808 08:28:20.354630 18496 heartbeater.cc:464] Registering TS with master...
I20260808 08:28:20.355094 18496 heartbeater.cc:511] Master 127.22.235.254:46859 requested a full tablet report, sending...
I20260808 08:28:20.356101 17985 ts_manager.cc:194] Registered new tserver with Master: 23f05ff8a1b04bb0a5a11717faec4639 (127.22.235.194:39403)
I20260808 08:28:20.356699 17985 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.194@KRBTEST.COM'} at 127.22.235.194:54987
W20260808 08:28:20.456053 18500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:28:20.456308 18500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:28:20.456360 18500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:28:20.460340 18500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:28:20.460398 18500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:28:20.460460 18500 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.195
I20260808 08:28:20.465816 18500 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:41847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab
--principal=kudu/127.22.235.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.195:34613
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.235.195
--webserver_port=33897
--tserver_master_addrs=127.22.235.254:46859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.195
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:28:20.466800 18500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:28:20.467761 18500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:28:20.477136 18509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:28:20.477157 18505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:28:20.477157 18507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:28:20.477242 18500 server_base.cc:1034] running on GCE node
I20260808 08:28:20.477738 18500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:28:20.478327 18500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:28:20.479496 18500 hybrid_clock.cc:648] HybridClock initialized: now 1786177700479467 us; error 38 us; skew 500 ppm
Aug 08 08:28:20 dist-test-slave-f7th krb5kdc[17045](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177700, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:28:20.482110 18500 init.cc:377] Logged in from keytab as kudu/127.22.235.195@KRBTEST.COM (short username kudu)
I20260808 08:28:20.482954 18500 webserver.cc:460] Webserver started at http://127.22.235.195:33897/ using document root <none> and password file <none>
I20260808 08:28:20.483487 18500 fs_manager.cc:359] Metadata directory not provided
I20260808 08:28:20.483568 18500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:28:20.487315 18500 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.001s
I20260808 08:28:20.489297 18517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:28:20.490418 18500 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 08:28:20.490571 18500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-2/wal
uuid: "622af630070e4d129fdbd0e90b1cfd84"
format_stamp: "Formatted at 2026-08-08 08:27:59 on dist-test-slave-f7th"
server_key: "c5bd78cf439e821a2bed4fde1abf1581"
server_key_iv: "95f36dcf00e9bfd3e03d6bec3c21cb74"
server_key_version: "encryptionkey@0"
I20260808 08:28:20.491148 18500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:28:20.503286 18500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:28:20.505854 18500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:28:20.506027 18500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:28:20.506577 18500 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:28:20.507404 18500 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:28:20.507452 18500 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:28:20.507503 18500 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:28:20.507531 18500 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:28:20.516305 18500 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.195:34613
I20260808 08:28:20.516333 18683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.195:34613 every 8 connection(s)
I20260808 08:28:20.517302 18500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
I20260808 08:28:20.518702 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 18500
Aug 08 08:28:20 dist-test-slave-f7th krb5kdc[17045](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177700, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.195@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:28:20.528409 18691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:20.519301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44184 (local address 127.22.235.254:46859)
0808 08:28:20.520413 (+ 1112us) server_negotiation.cc:206] Beginning negotiation
0808 08:28:20.520416 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:28:20.520428 (+ 12us) server_negotiation.cc:407] Connection header received
0808 08:28:20.520469 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:28:20.520471 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:28:20.520512 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:28:20.520587 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:28:20.521565 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:20.522107 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:20.523237 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:20.523585 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:20.523683 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:28:20.525164 (+ 1481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:28:20.525167 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:28:20.525169 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:28:20.525191 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:28:20.526634 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:28:20.527112 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:28:20.527116 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:28:20.527118 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:28:20.527203 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:28:20.527585 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:28:20.527589 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:28:20.527591 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:28:20.527763 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:28:20.527978 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0808 08:28:20.528127 (+ 149us) server_negotiation.cc:299] Negotiation successful
0808 08:28:20.528246 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1048,"thread_start_us":87,"threads_started":1}
I20260808 08:28:20.531311 18286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:20.519160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:40433 (local address 127.22.235.254:46859)
0808 08:28:20.519628 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0808 08:28:20.519634 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 08:28:20.520187 (+ 553us) server_negotiation.cc:407] Connection header received
0808 08:28:20.521686 (+ 1499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:28:20.521689 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:28:20.521737 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:28:20.521807 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:28:20.523154 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:20.523825 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:20.525344 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:20.525610 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:20.525713 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:28:20.528012 (+ 2299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:28:20.528015 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:28:20.528016 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:28:20.528037 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:28:20.529507 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:28:20.529961 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:28:20.529965 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:28:20.529966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:28:20.530045 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:28:20.530396 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:28:20.530399 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:28:20.530400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:28:20.530568 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:28:20.530639 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 08:28:20.531127 (+ 488us) server_negotiation.cc:299] Negotiation successful
0808 08:28:20.531208 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386}
I20260808 08:28:20.532105 18694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:20.519609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.195:40433)
0808 08:28:20.520055 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:20.520095 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 08:28:20.521485 (+ 1390us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:28:20.521979 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:28:20.521985 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:28:20.522396 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:28:20.523021 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:28:20.523040 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:28:20.523945 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:28:20.523949 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:28:20.525239 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:28:20.525249 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:28:20.525746 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:28:20.525749 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:28:20.525884 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:28:20.526353 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:28:20.526369 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:28:20.527864 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:28:20.529605 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:28:20.529610 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:28:20.529624 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:28:20.529866 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:28:20.530140 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:28:20.530142 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:28:20.530144 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:28:20.530316 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:28:20.530668 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:28:20.530673 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:28:20.530942 (+ 269us) client_negotiation.cc:769] Sending connection context
0808 08:28:20.531095 (+ 153us) client_negotiation.cc:240] Negotiation successful
0808 08:28:20.531341 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":111,"threads_started":1}
I20260808 08:28:20.533152 18687 heartbeater.cc:347] Connected to a master server at 127.22.235.254:46859
I20260808 08:28:20.533363 18687 heartbeater.cc:464] Registering TS with master...
I20260808 08:28:20.533787 18687 heartbeater.cc:511] Master 127.22.235.254:46859 requested a full tablet report, sending...
I20260808 08:28:20.534732 17985 ts_manager.cc:194] Registered new tserver with Master: 622af630070e4d129fdbd0e90b1cfd84 (127.22.235.195:34613)
I20260808 08:28:20.535246 17985 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.195@KRBTEST.COM'} at 127.22.235.195:40433
I20260808 08:28:20.541729 23471 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 08:28:20.545087 18286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:20.542259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44198 (local address 127.22.235.254:46859)
0808 08:28:20.542363 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0808 08:28:20.542368 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:28:20.542526 (+ 158us) server_negotiation.cc:407] Connection header received
0808 08:28:20.542653 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:28:20.542655 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:28:20.542740 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:28:20.542818 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:28:20.542997 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:20.543454 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:20.544195 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:20.544391 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:20.544455 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:28:20.544686 (+ 231us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:28:20.544809 (+ 123us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0808 08:28:20.545000 (+ 191us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44198: token expired
Metrics: {"server-negotiator.queue_time_us":37}
W20260808 08:28:20.545135 18286 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44198: token expired
W20260808 08:28:20.545449 17870 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.22.235.254:46859)
I20260808 08:28:20.551944 18286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:20.545768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44210 (local address 127.22.235.254:46859)
0808 08:28:20.545879 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 08:28:20.545882 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:28:20.545894 (+ 12us) server_negotiation.cc:407] Connection header received
0808 08:28:20.545994 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:28:20.545997 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:28:20.546035 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:28:20.546093 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:28:20.546656 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:20.547133 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:20.547880 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:20.548090 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:20.548171 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:28:20.549388 (+ 1217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:28:20.549392 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:28:20.549395 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:28:20.549419 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:28:20.550525 (+ 1106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:28:20.550897 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:28:20.550901 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:28:20.550903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:28:20.550978 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:28:20.551219 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:28:20.551223 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:28:20.551224 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:28:20.551381 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:28:20.551505 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 08:28:20.551713 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 08:28:20.551812 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 08:28:20.554164 17985 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44210:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 08:28:20.558118 17985 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 08:28:20.566021 18167 tablet_service.cc:1467] Processing CreateTablet for tablet e06544d83287436ca3694d3ce8f55658 (DEFAULT_TABLE table=test-table [id=bf8b759109db4318ac3a97510bdf0960]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:28:20.566721 18167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e06544d83287436ca3694d3ce8f55658. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:28:20.569732 18105 tablet_bootstrap.cc:492] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b: Bootstrap starting.
I20260808 08:28:20.570660 18105 tablet_bootstrap.cc:654] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b: Neither blocks nor log segments found. Creating new log.
I20260808 08:28:20.570710 18292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:20.564810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.195:34613 (local address 127.0.0.1:54982)
0808 08:28:20.564932 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:20.564946 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 08:28:20.565024 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:28:20.566226 (+ 1202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:28:20.566229 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:28:20.566247 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:28:20.566349 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:28:20.566355 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:28:20.567753 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:28:20.567756 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:28:20.568900 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:28:20.568911 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:28:20.570271 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:28:20.570274 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:28:20.570367 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:28:20.570409 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 08:28:20.570502 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 08:28:20.570586 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 08:28:20.570757 18707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:20.565201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54982 (local address 127.22.235.195:34613)
0808 08:28:20.565817 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0808 08:28:20.565828 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 08:28:20.565846 (+ 18us) server_negotiation.cc:407] Connection header received
0808 08:28:20.565907 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:28:20.565910 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:28:20.566057 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:28:20.566223 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:28:20.566450 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:20.567651 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:20.569029 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:20.570022 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:20.570279 (+ 257us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:28:20.570424 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 08:28:20.570534 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 08:28:20.570624 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":96,"threads_started":1}
I20260808 08:28:20.571321 18706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:20.565082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.194:39403 (local address 127.0.0.1:49062)
0808 08:28:20.565339 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:20.565352 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:28:20.565440 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:28:20.566436 (+ 996us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:28:20.566439 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:28:20.566459 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:28:20.566542 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:28:20.566548 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:28:20.568135 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:28:20.568139 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:28:20.569184 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:28:20.569193 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:28:20.570603 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:28:20.570607 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:28:20.570696 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:28:20.571038 (+ 342us) client_negotiation.cc:769] Sending connection context
0808 08:28:20.571124 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 08:28:20.571194 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":117,"threads_started":1}
I20260808 08:28:20.571358 18711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:20.565257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49062 (local address 127.22.235.194:39403)
0808 08:28:20.565700 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0808 08:28:20.565711 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 08:28:20.565735 (+ 24us) server_negotiation.cc:407] Connection header received
0808 08:28:20.565817 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:28:20.565823 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:28:20.566055 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:28:20.566243 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:28:20.566663 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:20.568011 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:20.569292 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:20.570301 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:20.570413 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:28:20.570536 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 08:28:20.571170 (+ 634us) server_negotiation.cc:299] Negotiation successful
0808 08:28:20.571251 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":148,"threads_started":1}
I20260808 08:28:20.572201 18105 tablet_bootstrap.cc:492] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b: No bootstrap required, opened a new log
I20260808 08:28:20.572288 18105 ts_tablet_manager.cc:1397] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260808 08:28:20.572742 18105 raft_consensus.cc:348] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "622af630070e4d129fdbd0e90b1cfd84" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34613 } } peers { permanent_uuid: "23f05ff8a1b04bb0a5a11717faec4639" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39403 } } peers { permanent_uuid: "5e6519a7db6245f9b459ee5905af464b" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45483 } }
I20260808 08:28:20.572832 18105 raft_consensus.cc:374] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:28:20.572858 18105 raft_consensus.cc:729] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e6519a7db6245f9b459ee5905af464b, State: Initialized, Role: FOLLOWER
I20260808 08:28:20.572953 18105 consensus_queue.cc:260] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "622af630070e4d129fdbd0e90b1cfd84" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34613 } } peers { permanent_uuid: "23f05ff8a1b04bb0a5a11717faec4639" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39403 } } peers { permanent_uuid: "5e6519a7db6245f9b459ee5905af464b" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45483 } }
I20260808 08:28:20.573185 18105 ts_tablet_manager.cc:1428] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 08:28:20.574671 18593 tablet_service.cc:1467] Processing CreateTablet for tablet e06544d83287436ca3694d3ce8f55658 (DEFAULT_TABLE table=test-table [id=bf8b759109db4318ac3a97510bdf0960]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:28:20.574671 18386 tablet_service.cc:1467] Processing CreateTablet for tablet e06544d83287436ca3694d3ce8f55658 (DEFAULT_TABLE table=test-table [id=bf8b759109db4318ac3a97510bdf0960]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:28:20.575778 18593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e06544d83287436ca3694d3ce8f55658. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:28:20.575773 18386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e06544d83287436ca3694d3ce8f55658. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:28:20.583784 18713 tablet_bootstrap.cc:492] T e06544d83287436ca3694d3ce8f55658 P 622af630070e4d129fdbd0e90b1cfd84: Bootstrap starting.
I20260808 08:28:20.584950 18714 tablet_bootstrap.cc:492] T e06544d83287436ca3694d3ce8f55658 P 23f05ff8a1b04bb0a5a11717faec4639: Bootstrap starting.
I20260808 08:28:20.586586 18713 tablet_bootstrap.cc:654] T e06544d83287436ca3694d3ce8f55658 P 622af630070e4d129fdbd0e90b1cfd84: Neither blocks nor log segments found. Creating new log.
I20260808 08:28:20.587469 18714 tablet_bootstrap.cc:654] T e06544d83287436ca3694d3ce8f55658 P 23f05ff8a1b04bb0a5a11717faec4639: Neither blocks nor log segments found. Creating new log.
I20260808 08:28:20.587530 18713 log.cc:824] T e06544d83287436ca3694d3ce8f55658 P 622af630070e4d129fdbd0e90b1cfd84: Log is configured to *not* fsync() on all Append() calls
I20260808 08:28:20.588261 18714 log.cc:824] T e06544d83287436ca3694d3ce8f55658 P 23f05ff8a1b04bb0a5a11717faec4639: Log is configured to *not* fsync() on all Append() calls
I20260808 08:28:20.589607 18713 tablet_bootstrap.cc:492] T e06544d83287436ca3694d3ce8f55658 P 622af630070e4d129fdbd0e90b1cfd84: No bootstrap required, opened a new log
I20260808 08:28:20.589787 18713 ts_tablet_manager.cc:1397] T e06544d83287436ca3694d3ce8f55658 P 622af630070e4d129fdbd0e90b1cfd84: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 08:28:20.590026 18714 tablet_bootstrap.cc:492] T e06544d83287436ca3694d3ce8f55658 P 23f05ff8a1b04bb0a5a11717faec4639: No bootstrap required, opened a new log
I20260808 08:28:20.590195 18714 ts_tablet_manager.cc:1397] T e06544d83287436ca3694d3ce8f55658 P 23f05ff8a1b04bb0a5a11717faec4639: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 08:28:20.592713 18713 raft_consensus.cc:348] T e06544d83287436ca3694d3ce8f55658 P 622af630070e4d129fdbd0e90b1cfd84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "622af630070e4d129fdbd0e90b1cfd84" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34613 } } peers { permanent_uuid: "23f05ff8a1b04bb0a5a11717faec4639" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39403 } } peers { permanent_uuid: "5e6519a7db6245f9b459ee5905af464b" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45483 } }
I20260808 08:28:20.592935 18713 raft_consensus.cc:374] T e06544d83287436ca3694d3ce8f55658 P 622af630070e4d129fdbd0e90b1cfd84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:28:20.592986 18713 raft_consensus.cc:729] T e06544d83287436ca3694d3ce8f55658 P 622af630070e4d129fdbd0e90b1cfd84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 622af630070e4d129fdbd0e90b1cfd84, State: Initialized, Role: FOLLOWER
I20260808 08:28:20.592953 18714 raft_consensus.cc:348] T e06544d83287436ca3694d3ce8f55658 P 23f05ff8a1b04bb0a5a11717faec4639 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "622af630070e4d129fdbd0e90b1cfd84" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34613 } } peers { permanent_uuid: "23f05ff8a1b04bb0a5a11717faec4639" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39403 } } peers { permanent_uuid: "5e6519a7db6245f9b459ee5905af464b" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45483 } }
I20260808 08:28:20.593201 18714 raft_consensus.cc:374] T e06544d83287436ca3694d3ce8f55658 P 23f05ff8a1b04bb0a5a11717faec4639 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:28:20.593262 18714 raft_consensus.cc:729] T e06544d83287436ca3694d3ce8f55658 P 23f05ff8a1b04bb0a5a11717faec4639 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23f05ff8a1b04bb0a5a11717faec4639, State: Initialized, Role: FOLLOWER
I20260808 08:28:20.593482 18713 consensus_queue.cc:260] T e06544d83287436ca3694d3ce8f55658 P 622af630070e4d129fdbd0e90b1cfd84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "622af630070e4d129fdbd0e90b1cfd84" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34613 } } peers { permanent_uuid: "23f05ff8a1b04bb0a5a11717faec4639" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39403 } } peers { permanent_uuid: "5e6519a7db6245f9b459ee5905af464b" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45483 } }
I20260808 08:28:20.593639 18714 consensus_queue.cc:260] T e06544d83287436ca3694d3ce8f55658 P 23f05ff8a1b04bb0a5a11717faec4639 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "622af630070e4d129fdbd0e90b1cfd84" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34613 } } peers { permanent_uuid: "23f05ff8a1b04bb0a5a11717faec4639" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39403 } } peers { permanent_uuid: "5e6519a7db6245f9b459ee5905af464b" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45483 } }
I20260808 08:28:20.594143 18687 heartbeater.cc:503] Master 127.22.235.254:46859 was elected leader, sending a full tablet report...
I20260808 08:28:20.594226 18496 heartbeater.cc:503] Master 127.22.235.254:46859 was elected leader, sending a full tablet report...
I20260808 08:28:20.594472 18713 ts_tablet_manager.cc:1428] T e06544d83287436ca3694d3ce8f55658 P 622af630070e4d129fdbd0e90b1cfd84: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 08:28:20.594527 18714 ts_tablet_manager.cc:1428] T e06544d83287436ca3694d3ce8f55658 P 23f05ff8a1b04bb0a5a11717faec4639: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260808 08:28:20.639619 18724 raft_consensus.cc:482] T e06544d83287436ca3694d3ce8f55658 P 622af630070e4d129fdbd0e90b1cfd84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 08:28:20.639926 18724 raft_consensus.cc:504] T e06544d83287436ca3694d3ce8f55658 P 622af630070e4d129fdbd0e90b1cfd84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "622af630070e4d129fdbd0e90b1cfd84" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34613 } } peers { permanent_uuid: "23f05ff8a1b04bb0a5a11717faec4639" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39403 } } peers { permanent_uuid: "5e6519a7db6245f9b459ee5905af464b" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45483 } }
I20260808 08:28:20.641147 18724 leader_election.cc:288] T e06544d83287436ca3694d3ce8f55658 P 622af630070e4d129fdbd0e90b1cfd84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23f05ff8a1b04bb0a5a11717faec4639 (127.22.235.194:39403), 5e6519a7db6245f9b459ee5905af464b (127.22.235.193:45483)
I20260808 08:28:20.644928 18694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:20.641393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.194:39403 (local address 127.22.235.195:47019)
0808 08:28:20.641531 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:20.641546 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 08:28:20.641654 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:28:20.642041 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:28:20.642045 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:28:20.642066 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:28:20.642178 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:28:20.642185 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:28:20.642994 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:28:20.642998 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:28:20.643825 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:28:20.643835 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:28:20.644484 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:28:20.644488 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:28:20.644606 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:28:20.644650 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 08:28:20.644722 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 08:28:20.644795 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 08:28:20.644940 18711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:20.641602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:47019 (local address 127.22.235.194:39403)
0808 08:28:20.641741 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 08:28:20.641745 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:28:20.641761 (+ 16us) server_negotiation.cc:407] Connection header received
0808 08:28:20.641822 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:28:20.641825 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:28:20.641876 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:28:20.641956 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:28:20.642306 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:20.642881 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:20.643957 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:20.644356 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:20.644463 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:28:20.644524 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 08:28:20.644745 (+ 221us) server_negotiation.cc:299] Negotiation successful
0808 08:28:20.644831 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 08:28:20.645560 18726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:20.641690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:45483 (local address 127.22.235.195:53769)
0808 08:28:20.642229 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:20.642241 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 08:28:20.642327 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:28:20.642686 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:28:20.642689 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:28:20.642707 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:28:20.642865 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:28:20.642872 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:28:20.643709 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:28:20.643712 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:28:20.644452 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:28:20.644461 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:28:20.645183 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:28:20.645187 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:28:20.645272 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:28:20.645293 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 08:28:20.645352 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 08:28:20.645417 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":159,"threads_started":1}
I20260808 08:28:20.645547 18294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:20.641926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:53769 (local address 127.22.235.193:45483)
0808 08:28:20.642053 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 08:28:20.642057 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:28:20.642311 (+ 254us) server_negotiation.cc:407] Connection header received
0808 08:28:20.642499 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:28:20.642503 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:28:20.642567 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:28:20.642659 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:28:20.642993 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:20.643599 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:20.644576 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:20.645040 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:20.645157 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:28:20.645327 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 08:28:20.645375 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 08:28:20.645440 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 08:28:20.645547 18426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e06544d83287436ca3694d3ce8f55658" candidate_uuid: "622af630070e4d129fdbd0e90b1cfd84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23f05ff8a1b04bb0a5a11717faec4639" is_pre_election: true
I20260808 08:28:20.645874 18426 raft_consensus.cc:2454] T e06544d83287436ca3694d3ce8f55658 P 23f05ff8a1b04bb0a5a11717faec4639 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 622af630070e4d129fdbd0e90b1cfd84 in term 0.
I20260808 08:28:20.646160 18210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e06544d83287436ca3694d3ce8f55658" candidate_uuid: "622af630070e4d129fdbd0e90b1cfd84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e6519a7db6245f9b459ee5905af464b" is_pre_election: true
I20260808 08:28:20.646377 18519 leader_election.cc:302] T e06544d83287436ca3694d3ce8f55658 P 622af630070e4d129fdbd0e90b1cfd84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23f05ff8a1b04bb0a5a11717faec4639, 622af630070e4d129fdbd0e90b1cfd84; no voters:
I20260808 08:28:20.646431 18210 raft_consensus.cc:2454] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 622af630070e4d129fdbd0e90b1cfd84 in term 0.
I20260808 08:28:20.646637 18724 raft_consensus.cc:2781] T e06544d83287436ca3694d3ce8f55658 P 622af630070e4d129fdbd0e90b1cfd84 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 08:28:20.646724 18724 raft_consensus.cc:482] T e06544d83287436ca3694d3ce8f55658 P 622af630070e4d129fdbd0e90b1cfd84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 08:28:20.646775 18724 raft_consensus.cc:3037] T e06544d83287436ca3694d3ce8f55658 P 622af630070e4d129fdbd0e90b1cfd84 [term 0 FOLLOWER]: Advancing to term 1
W20260808 08:28:20.647589 18283 tablet.cc:2369] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 08:28:20.648005 18724 raft_consensus.cc:504] T e06544d83287436ca3694d3ce8f55658 P 622af630070e4d129fdbd0e90b1cfd84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "622af630070e4d129fdbd0e90b1cfd84" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34613 } } peers { permanent_uuid: "23f05ff8a1b04bb0a5a11717faec4639" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39403 } } peers { permanent_uuid: "5e6519a7db6245f9b459ee5905af464b" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45483 } }
I20260808 08:28:20.648296 18724 leader_election.cc:288] T e06544d83287436ca3694d3ce8f55658 P 622af630070e4d129fdbd0e90b1cfd84 [CANDIDATE]: Term 1 election: Requested vote from peers 23f05ff8a1b04bb0a5a11717faec4639 (127.22.235.194:39403), 5e6519a7db6245f9b459ee5905af464b (127.22.235.193:45483)
I20260808 08:28:20.648641 18426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e06544d83287436ca3694d3ce8f55658" candidate_uuid: "622af630070e4d129fdbd0e90b1cfd84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23f05ff8a1b04bb0a5a11717faec4639"
I20260808 08:28:20.648756 18426 raft_consensus.cc:3037] T e06544d83287436ca3694d3ce8f55658 P 23f05ff8a1b04bb0a5a11717faec4639 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:28:20.648743 18210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e06544d83287436ca3694d3ce8f55658" candidate_uuid: "622af630070e4d129fdbd0e90b1cfd84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e6519a7db6245f9b459ee5905af464b"
I20260808 08:28:20.648934 18210 raft_consensus.cc:3037] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:28:20.649775 18426 raft_consensus.cc:2454] T e06544d83287436ca3694d3ce8f55658 P 23f05ff8a1b04bb0a5a11717faec4639 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 622af630070e4d129fdbd0e90b1cfd84 in term 1.
I20260808 08:28:20.650008 18210 raft_consensus.cc:2454] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 622af630070e4d129fdbd0e90b1cfd84 in term 1.
I20260808 08:28:20.650053 18519 leader_election.cc:302] T e06544d83287436ca3694d3ce8f55658 P 622af630070e4d129fdbd0e90b1cfd84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23f05ff8a1b04bb0a5a11717faec4639, 622af630070e4d129fdbd0e90b1cfd84; no voters:
I20260808 08:28:20.650197 18724 raft_consensus.cc:2781] T e06544d83287436ca3694d3ce8f55658 P 622af630070e4d129fdbd0e90b1cfd84 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:28:20.650405 18724 raft_consensus.cc:686] T e06544d83287436ca3694d3ce8f55658 P 622af630070e4d129fdbd0e90b1cfd84 [term 1 LEADER]: Becoming Leader. State: Replica: 622af630070e4d129fdbd0e90b1cfd84, State: Running, Role: LEADER
I20260808 08:28:20.650701 18724 consensus_queue.cc:237] T e06544d83287436ca3694d3ce8f55658 P 622af630070e4d129fdbd0e90b1cfd84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "622af630070e4d129fdbd0e90b1cfd84" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34613 } } peers { permanent_uuid: "23f05ff8a1b04bb0a5a11717faec4639" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39403 } } peers { permanent_uuid: "5e6519a7db6245f9b459ee5905af464b" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45483 } }
I20260808 08:28:20.653908 17982 catalog_manager.cc:5495] T e06544d83287436ca3694d3ce8f55658 P 622af630070e4d129fdbd0e90b1cfd84 reported cstate change: term changed from 0 to 1, leader changed from <none> to 622af630070e4d129fdbd0e90b1cfd84 (127.22.235.195). New cstate: current_term: 1 leader_uuid: "622af630070e4d129fdbd0e90b1cfd84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "622af630070e4d129fdbd0e90b1cfd84" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "23f05ff8a1b04bb0a5a11717faec4639" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e6519a7db6245f9b459ee5905af464b" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45483 } health_report { overall_health: UNKNOWN } } }
I20260808 08:28:20.669991 18707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:20.666817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39134 (local address 127.22.235.195:34613)
0808 08:28:20.666947 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 08:28:20.666950 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:28:20.666963 (+ 13us) server_negotiation.cc:407] Connection header received
0808 08:28:20.667101 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:28:20.667104 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:28:20.667156 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:28:20.667246 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:28:20.667424 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:20.667952 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:20.668850 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:20.669064 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:20.669151 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:28:20.669347 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:28:20.669725 (+ 378us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 08:28:20.669818 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 08:28:20.669842 (+ 24us) server_negotiation.cc:299] Negotiation successful
0808 08:28:20.669890 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 08:28:20.676091 18210 raft_consensus.cc:1261] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b [term 1 FOLLOWER]: Refusing update from remote peer 622af630070e4d129fdbd0e90b1cfd84: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:28:20.676137 18426 raft_consensus.cc:1261] T e06544d83287436ca3694d3ce8f55658 P 23f05ff8a1b04bb0a5a11717faec4639 [term 1 FOLLOWER]: Refusing update from remote peer 622af630070e4d129fdbd0e90b1cfd84: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:28:20.676779 18729 consensus_queue.cc:1035] T e06544d83287436ca3694d3ce8f55658 P 622af630070e4d129fdbd0e90b1cfd84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e6519a7db6245f9b459ee5905af464b" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:28:20.677007 18724 consensus_queue.cc:1035] T e06544d83287436ca3694d3ce8f55658 P 622af630070e4d129fdbd0e90b1cfd84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23f05ff8a1b04bb0a5a11717faec4639" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:28:20.683980 18737 mvcc.cc:205] Tried to move back new op lower bound from 7316183861961842688 to 7316183861867753472. Current Snapshot: MvccSnapshot[applied={T|T < 7316183861961842688}]
I20260808 08:28:20.692291 17985 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44210:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:28:20.692433 17985 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44210:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:28:20.694595 17985 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676: Sending DeleteTablet for 3 replicas of tablet e06544d83287436ca3694d3ce8f55658
I20260808 08:28:20.695340 18593 tablet_service.cc:1514] Processing DeleteTablet for tablet e06544d83287436ca3694d3ce8f55658 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:28:20 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:54982
I20260808 08:28:20.695348 18386 tablet_service.cc:1514] Processing DeleteTablet for tablet e06544d83287436ca3694d3ce8f55658 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:28:20 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:49062
I20260808 08:28:20.695433 18167 tablet_service.cc:1514] Processing DeleteTablet for tablet e06544d83287436ca3694d3ce8f55658 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:28:20 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:37148
I20260808 08:28:20.695748 18750 tablet_replica.cc:330] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b: stopping tablet replica
I20260808 08:28:20.695818 18752 tablet_replica.cc:330] T e06544d83287436ca3694d3ce8f55658 P 23f05ff8a1b04bb0a5a11717faec4639: stopping tablet replica
I20260808 08:28:20.695895 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 18039
I20260808 08:28:20.695911 18750 raft_consensus.cc:2229] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 08:28:20.704785 18751 tablet_replica.cc:330] T e06544d83287436ca3694d3ce8f55658 P 622af630070e4d129fdbd0e90b1cfd84: stopping tablet replica
I20260808 08:28:20.705087 18751 raft_consensus.cc:2229] T e06544d83287436ca3694d3ce8f55658 P 622af630070e4d129fdbd0e90b1cfd84 [term 1 LEADER]: Raft consensus shutting down.
I20260808 08:28:20.705442 18751 raft_consensus.cc:2258] T e06544d83287436ca3694d3ce8f55658 P 622af630070e4d129fdbd0e90b1cfd84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:28:20.696159 18752 raft_consensus.cc:2229] T e06544d83287436ca3694d3ce8f55658 P 23f05ff8a1b04bb0a5a11717faec4639 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 08:28:20.713964 18752 raft_consensus.cc:2258] T e06544d83287436ca3694d3ce8f55658 P 23f05ff8a1b04bb0a5a11717faec4639 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:28:20.716868 18752 ts_tablet_manager.cc:1905] T e06544d83287436ca3694d3ce8f55658 P 23f05ff8a1b04bb0a5a11717faec4639: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:28:20.720455 18752 ts_tablet_manager.cc:1918] T e06544d83287436ca3694d3ce8f55658 P 23f05ff8a1b04bb0a5a11717faec4639: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 08:28:20.720592 18752 log.cc:1196] T e06544d83287436ca3694d3ce8f55658 P 23f05ff8a1b04bb0a5a11717faec4639: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-1/wal/wals/e06544d83287436ca3694d3ce8f55658
I20260808 08:28:20.721069 18752 ts_tablet_manager.cc:1939] T e06544d83287436ca3694d3ce8f55658 P 23f05ff8a1b04bb0a5a11717faec4639: Deleting consensus metadata
I20260808 08:28:20.725523 17955 catalog_manager.cc:4841] TS 23f05ff8a1b04bb0a5a11717faec4639 (127.22.235.194:39403): tablet e06544d83287436ca3694d3ce8f55658 (table test-table [id=bf8b759109db4318ac3a97510bdf0960]) successfully deleted
I20260808 08:28:20.696154 18750 raft_consensus.cc:2258] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:28:20.727133 18751 ts_tablet_manager.cc:1905] T e06544d83287436ca3694d3ce8f55658 P 622af630070e4d129fdbd0e90b1cfd84: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:28:20.729835 18751 ts_tablet_manager.cc:1918] T e06544d83287436ca3694d3ce8f55658 P 622af630070e4d129fdbd0e90b1cfd84: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 08:28:20.729934 18751 log.cc:1196] T e06544d83287436ca3694d3ce8f55658 P 622af630070e4d129fdbd0e90b1cfd84: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-2/wal/wals/e06544d83287436ca3694d3ce8f55658
I20260808 08:28:20.730301 18751 ts_tablet_manager.cc:1939] T e06544d83287436ca3694d3ce8f55658 P 622af630070e4d129fdbd0e90b1cfd84: Deleting consensus metadata
I20260808 08:28:20.731559 17955 catalog_manager.cc:4841] TS 622af630070e4d129fdbd0e90b1cfd84 (127.22.235.195:34613): tablet e06544d83287436ca3694d3ce8f55658 (table test-table [id=bf8b759109db4318ac3a97510bdf0960]) successfully deleted
W20260808 08:28:20.733851 17955 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.235.193:45483 (error 108)
I20260808 08:28:20.734113 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 18288
I20260808 08:28:20.739934 18706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:20.739602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:45483 (local address 127.0.0.1:34602)
0808 08:28:20.739758 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:20.739841 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.193:45483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 08:28:20.740140 17955 catalog_manager.cc:4584] TS 5e6519a7db6245f9b459ee5905af464b (127.22.235.193:45483): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e06544d83287436ca3694d3ce8f55658: Network error: Client connection negotiation failed: client connection to 127.22.235.193:45483: connect: Connection refused (error 111)
I20260808 08:28:20.744199 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 18500
I20260808 08:28:20.755069 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 17930
I20260808 08:28:40.766429 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:46859
--webserver_interface=127.22.235.254
--webserver_port=34107
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:41847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:46859
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:28:40.884114 18755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:28:40.884454 18755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:28:40.884522 18755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:28:40.888430 18755 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 08:28:40.888476 18755 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 08:28:40.888490 18755 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 08:28:40.888504 18755 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:28:40.888520 18755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:28:40.888552 18755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:28:40.888581 18755 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:28:40.888602 18755 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:28:40.893731 18755 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:41847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:46859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:46859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.235.254
--webserver_port=34107
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18755
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:28:40.894724 18755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:28:40.895783 18755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:28:40.904403 18763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:28:40.904352 18755 server_base.cc:1034] running on GCE node
W20260808 08:28:40.904475 18766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:28:40.904585 18764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:28:40.905175 18755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:28:40.906257 18755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:28:40.907501 18755 hybrid_clock.cc:648] HybridClock initialized: now 1786177720907443 us; error 29 us; skew 500 ppm
Aug 08 08:28:40 dist-test-slave-f7th krb5kdc[17045](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177720, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:28:40.910434 18755 init.cc:377] Logged in from keytab as kudu/127.22.235.254@KRBTEST.COM (short username kudu)
I20260808 08:28:40.911353 18755 webserver.cc:460] Webserver started at http://127.22.235.254:34107/ using document root <none> and password file <none>
I20260808 08:28:40.911899 18755 fs_manager.cc:359] Metadata directory not provided
I20260808 08:28:40.911969 18755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:28:40.915917 18755 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:28:40.918068 18775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:28:40.919250 18755 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260808 08:28:40.919370 18755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "955105d3eda549ddb05259d3c312c676"
format_stamp: "Formatted at 2026-08-08 08:27:58 on dist-test-slave-f7th"
server_key: "14611b503cf030b60a6e621bfa39d2b0"
server_key_iv: "b669d78e0ca36526072a5faa4123b6f3"
server_key_version: "encryptionkey@0"
I20260808 08:28:40.919888 18755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:28:40.937458 18755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:28:40.940587 18755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:28:40.940817 18755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:28:40.947819 18755 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.254:46859
I20260808 08:28:40.948082 18861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.254:46859 every 8 connection(s)
I20260808 08:28:40.949016 18755 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:28:40.953375 18865 sys_catalog.cc:263] Verifying existing consensus state
I20260808 08:28:40.954663 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 18755
I20260808 08:28:40.955124 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.193:45483
--local_ip_for_outbound_sockets=127.22.235.193
--tserver_master_addrs=127.22.235.254:46859
--webserver_port=32955
--webserver_interface=127.22.235.193
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab
--principal=kudu/127.22.235.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:41847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 08:28:40.955211 18865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676: Bootstrap starting.
I20260808 08:28:40.972532 18865 log.cc:824] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676: Log is configured to *not* fsync() on all Append() calls
I20260808 08:28:40.983692 18865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 08:28:40.984057 18865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676: Bootstrap complete.
I20260808 08:28:40.988014 18865 raft_consensus.cc:348] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "955105d3eda549ddb05259d3c312c676" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46859 } }
I20260808 08:28:40.988389 18865 raft_consensus.cc:729] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 955105d3eda549ddb05259d3c312c676, State: Initialized, Role: FOLLOWER
I20260808 08:28:40.988898 18865 consensus_queue.cc:260] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "955105d3eda549ddb05259d3c312c676" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46859 } }
I20260808 08:28:40.989020 18865 raft_consensus.cc:388] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:28:40.989089 18865 raft_consensus.cc:482] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:28:40.989182 18865 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [term 2 FOLLOWER]: Advancing to term 3
I20260808 08:28:40.991174 18865 raft_consensus.cc:504] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "955105d3eda549ddb05259d3c312c676" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46859 } }
I20260808 08:28:40.991483 18865 leader_election.cc:302] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 955105d3eda549ddb05259d3c312c676; no voters:
I20260808 08:28:40.991786 18865 leader_election.cc:288] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 08:28:40.991883 18874 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 08:28:40.992100 18874 raft_consensus.cc:686] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [term 3 LEADER]: Becoming Leader. State: Replica: 955105d3eda549ddb05259d3c312c676, State: Running, Role: LEADER
I20260808 08:28:40.992451 18874 consensus_queue.cc:237] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "955105d3eda549ddb05259d3c312c676" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46859 } }
I20260808 08:28:40.993088 18865 sys_catalog.cc:564] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:28:40.993985 18877 sys_catalog.cc:455] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 955105d3eda549ddb05259d3c312c676. Latest consensus state: current_term: 3 leader_uuid: "955105d3eda549ddb05259d3c312c676" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "955105d3eda549ddb05259d3c312c676" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46859 } } }
I20260808 08:28:40.994113 18877 sys_catalog.cc:458] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [sys.catalog]: This master's current role is: LEADER
I20260808 08:28:40.994447 18886 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:28:40.994441 18876 sys_catalog.cc:455] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "955105d3eda549ddb05259d3c312c676" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "955105d3eda549ddb05259d3c312c676" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46859 } } }
I20260808 08:28:40.994592 18876 sys_catalog.cc:458] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676 [sys.catalog]: This master's current role is: LEADER
I20260808 08:28:40.999516 18886 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:28:41.000226 18886 catalog_manager.cc:1259] Loaded cluster ID: 01abe1afe9464a23bca90e7f4fed11c1
I20260808 08:28:41.000279 18886 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:28:41.002058 18886 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:28:41.002638 18886 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676: Loaded TSK: 0
I20260808 08:28:41.002705 18886 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 955105d3eda549ddb05259d3c312c676: Loaded TSK: 1
F20260808 08:28:41.003306 18886 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f6032e599ad google::LogMessage::Fail() at ??:0
@ 0x7f6032e5c88a google::LogMessage::SendToLog() at ??:0
@ 0x7f6032e594de google::LogMessage::Flush() at ??:0
@ 0x7f6032e5aa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f6035b1d18d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f6035b15f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f603d7c27bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f603d79c99c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f603d79d178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f603d7dedf8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f603d7fd82b std::function<>::operator()() at ??:0
@ 0x7f603d79cb4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f603d79df3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f603d79a579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f603d7de640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f60350ab3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f60350abc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f60350ad3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f603509c512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f60366e06db start_thread at ??:0
@ 0x7f603195761f clone at ??:0
W20260808 08:28:41.100788 18871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:28:41.101053 18871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:28:41.101107 18871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:28:41.105130 18871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:28:41.105180 18871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:28:41.105248 18871 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.193
I20260808 08:28:41.110445 18871 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:41847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab
--principal=kudu/127.22.235.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.193:45483
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.235.193
--webserver_port=32955
--tserver_master_addrs=127.22.235.254:46859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18871
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.193
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:28:41.111526 18871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:28:41.112540 18871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:28:41.122043 18948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:28:41.122220 18871 server_base.cc:1034] running on GCE node
W20260808 08:28:41.122056 18947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:28:41.122143 18952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:28:41.122882 18871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:28:41.123615 18871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:28:41.124799 18871 hybrid_clock.cc:648] HybridClock initialized: now 1786177721124766 us; error 40 us; skew 500 ppm
Aug 08 08:28:41 dist-test-slave-f7th krb5kdc[17045](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177721, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:28:41.128113 18871 init.cc:377] Logged in from keytab as kudu/127.22.235.193@KRBTEST.COM (short username kudu)
I20260808 08:28:41.129205 18871 webserver.cc:460] Webserver started at http://127.22.235.193:32955/ using document root <none> and password file <none>
I20260808 08:28:41.129884 18871 fs_manager.cc:359] Metadata directory not provided
I20260808 08:28:41.129973 18871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:28:41.135003 18871 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 08:28:41.137638 18961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:28:41.139104 18871 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260808 08:28:41.139223 18871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-0/wal
uuid: "5e6519a7db6245f9b459ee5905af464b"
format_stamp: "Formatted at 2026-08-08 08:27:58 on dist-test-slave-f7th"
server_key: "19f824c19d97b739b738edbccf4247cd"
server_key_iv: "7d3c06265e9b909dba1c2f01ae18a614"
server_key_version: "encryptionkey@0"
I20260808 08:28:41.139879 18871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:28:41.158923 18871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:28:41.161945 18871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:28:41.162125 18871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:28:41.162685 18871 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:28:41.163765 18972 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 08:28:41.169260 18871 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 08:28:41.169319 18871 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260808 08:28:41.169391 18871 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 08:28:41.171012 18871 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 08:28:41.171057 18871 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260808 08:28:41.171383 18972 tablet_bootstrap.cc:492] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b: Bootstrap starting.
I20260808 08:28:41.181129 18871 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.193:45483
I20260808 08:28:41.181370 19149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.193:45483 every 8 connection(s)
I20260808 08:28:41.182324 18871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
I20260808 08:28:41.182857 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 18871
I20260808 08:28:41.183323 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.194:39403
--local_ip_for_outbound_sockets=127.22.235.194
--tserver_master_addrs=127.22.235.254:46859
--webserver_port=43027
--webserver_interface=127.22.235.194
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab
--principal=kudu/127.22.235.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:41847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 08:28:41.193671 18972 log.cc:824] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b: Log is configured to *not* fsync() on all Append() calls
I20260808 08:28:41.202658 18972 tablet_bootstrap.cc:492] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 08:28:41.203069 18972 tablet_bootstrap.cc:492] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b: Bootstrap complete.
I20260808 08:28:41.204411 18972 ts_tablet_manager.cc:1397] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b: Time spent bootstrapping tablet: real 0.033s user 0.024s sys 0.004s
I20260808 08:28:41.207767 18972 raft_consensus.cc:348] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "622af630070e4d129fdbd0e90b1cfd84" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34613 } } peers { permanent_uuid: "23f05ff8a1b04bb0a5a11717faec4639" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39403 } } peers { permanent_uuid: "5e6519a7db6245f9b459ee5905af464b" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45483 } }
I20260808 08:28:41.208316 18972 raft_consensus.cc:729] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e6519a7db6245f9b459ee5905af464b, State: Initialized, Role: FOLLOWER
I20260808 08:28:41.208930 18972 consensus_queue.cc:260] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "622af630070e4d129fdbd0e90b1cfd84" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34613 } } peers { permanent_uuid: "23f05ff8a1b04bb0a5a11717faec4639" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39403 } } peers { permanent_uuid: "5e6519a7db6245f9b459ee5905af464b" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45483 } }
I20260808 08:28:41.210019 18972 ts_tablet_manager.cc:1428] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b: Time spent starting tablet: real 0.006s user 0.005s sys 0.004s
W20260808 08:28:41.228389 19152 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:28:41.184548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.193:39551)
0808 08:28:41.185763 (+ 1215us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:41.185821 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 08:28:41.186834 (+ 1013us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:28:41.227408 (+ 40574us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":990,"thread_start_us":138,"threads_started":1}
W20260808 08:28:41.228947 19150 heartbeater.cc:644] Failed to heartbeat to 127.22.235.254:46859 (0 consecutive failures): Network error: Failed to ping master at 127.22.235.254:46859: Client connection negotiation failed: client connection to 127.22.235.254:46859: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 08:28:41.229734 19152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:41.229406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.193:39949)
0808 08:28:41.229569 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:41.229658 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 08:28:41.230471 19152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:41.230267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.193:36639)
0808 08:28:41.230373 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:41.230420 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 08:28:41.230713 19150 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 08:28:41.303980 19155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:28:41.304231 19155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:28:41.304293 19155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:28:41.308214 19155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:28:41.308274 19155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:28:41.308352 19155 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.194
I20260808 08:28:41.313498 19155 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:41847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab
--principal=kudu/127.22.235.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.194:39403
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.235.194
--webserver_port=43027
--tserver_master_addrs=127.22.235.254:46859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19155
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.194
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:28:41.314538 19155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:28:41.315521 19155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:28:41.324810 19166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:28:41.324831 19163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:28:41.324842 19164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:28:41.325398 19155 server_base.cc:1034] running on GCE node
I20260808 08:28:41.325747 19155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:28:41.326326 19155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:28:41.327491 19155 hybrid_clock.cc:648] HybridClock initialized: now 1786177721327474 us; error 32 us; skew 500 ppm
Aug 08 08:28:41 dist-test-slave-f7th krb5kdc[17045](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177721, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:28:41.330193 19155 init.cc:377] Logged in from keytab as kudu/127.22.235.194@KRBTEST.COM (short username kudu)
I20260808 08:28:41.331032 19155 webserver.cc:460] Webserver started at http://127.22.235.194:43027/ using document root <none> and password file <none>
I20260808 08:28:41.331543 19155 fs_manager.cc:359] Metadata directory not provided
I20260808 08:28:41.331602 19155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:28:41.335300 19155 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:28:41.337296 19176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:28:41.338534 19155 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:28:41.338631 19155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-1/wal
uuid: "23f05ff8a1b04bb0a5a11717faec4639"
format_stamp: "Formatted at 2026-08-08 08:27:59 on dist-test-slave-f7th"
server_key: "d5d81801df520fb0567eb4182667b79a"
server_key_iv: "a19e140047c133a8959da581b505119c"
server_key_version: "encryptionkey@0"
I20260808 08:28:41.339201 19155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:28:41.351574 19155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:28:41.354223 19155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:28:41.354369 19155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:28:41.354884 19155 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:28:41.355679 19155 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:28:41.355726 19155 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:28:41.355764 19155 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:28:41.355784 19155 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:28:41.364367 19155 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.194:39403
I20260808 08:28:41.364391 19358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.194:39403 every 8 connection(s)
I20260808 08:28:41.365365 19155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
I20260808 08:28:41.369025 19362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:41.367380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.194:34721)
0808 08:28:41.367832 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:41.368264 (+ 432us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":119,"threads_started":1}
W20260808 08:28:41.369530 19359 heartbeater.cc:644] Failed to heartbeat to 127.22.235.254:46859 (0 consecutive failures): Network error: Failed to ping master at 127.22.235.254:46859: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
I20260808 08:28:41.370072 19362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:41.369855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.194:43911)
0808 08:28:41.369972 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:41.370020 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 08:28:41.370606 19362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:41.370444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.194:48347)
0808 08:28:41.370524 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:41.370564 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 08:28:41.370746 19359 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 08:28:41.373642 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 19155
I20260808 08:28:41.374017 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.195:34613
--local_ip_for_outbound_sockets=127.22.235.195
--tserver_master_addrs=127.22.235.254:46859
--webserver_port=33897
--webserver_interface=127.22.235.195
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab
--principal=kudu/127.22.235.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:41847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:28:41.486135 19363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:28:41.486394 19363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:28:41.486435 19363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:28:41.490357 19363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:28:41.490413 19363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:28:41.490479 19363 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.195
I20260808 08:28:41.495580 19363 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:41847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab
--principal=kudu/127.22.235.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.195:34613
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.235.195
--webserver_port=33897
--tserver_master_addrs=127.22.235.254:46859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.195
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:28:41.496572 19363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:28:41.497581 19363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:28:41.506943 19376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:28:41.506987 19373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:28:41.506943 19372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:28:41.507215 19363 server_base.cc:1034] running on GCE node
I20260808 08:28:41.507750 19363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:28:41.508364 19363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:28:41.509524 19363 hybrid_clock.cc:648] HybridClock initialized: now 1786177721509500 us; error 37 us; skew 500 ppm
Aug 08 08:28:41 dist-test-slave-f7th krb5kdc[17045](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177721, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:28:41.512154 19363 init.cc:377] Logged in from keytab as kudu/127.22.235.195@KRBTEST.COM (short username kudu)
I20260808 08:28:41.513048 19363 webserver.cc:460] Webserver started at http://127.22.235.195:33897/ using document root <none> and password file <none>
I20260808 08:28:41.513628 19363 fs_manager.cc:359] Metadata directory not provided
I20260808 08:28:41.513693 19363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:28:41.517458 19363 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:28:41.519534 19389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:28:41.520777 19363 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 08:28:41.520876 19363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-2/wal
uuid: "622af630070e4d129fdbd0e90b1cfd84"
format_stamp: "Formatted at 2026-08-08 08:27:59 on dist-test-slave-f7th"
server_key: "c5bd78cf439e821a2bed4fde1abf1581"
server_key_iv: "95f36dcf00e9bfd3e03d6bec3c21cb74"
server_key_version: "encryptionkey@0"
I20260808 08:28:41.521379 19363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:28:41.537281 19363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:28:41.540117 19363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:28:41.540284 19363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:28:41.540925 19363 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:28:41.541759 19363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:28:41.541811 19363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:28:41.541867 19363 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:28:41.541896 19363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:28:41.550740 19363 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.195:34613
I20260808 08:28:41.550804 19562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.195:34613 every 8 connection(s)
I20260808 08:28:41.551800 19363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
I20260808 08:28:41.555541 19569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:41.553969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.195:38181)
0808 08:28:41.554420 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:41.554840 (+ 420us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":113,"threads_started":1}
W20260808 08:28:41.555958 19564 heartbeater.cc:644] Failed to heartbeat to 127.22.235.254:46859 (0 consecutive failures): Network error: Failed to ping master at 127.22.235.254:46859: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
I20260808 08:28:41.556532 19569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:41.556338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.195:57321)
0808 08:28:41.556431 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:41.556477 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 08:28:41.557104 19569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:41.556938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.195:44827)
0808 08:28:41.557023 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:41.557062 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 08:28:41.557277 19564 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 08:28:41.561141 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 19363
I20260808 08:28:42.231732 19573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:42.231301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.193:52125)
0808 08:28:42.231561 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:42.231616 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":75,"threads_started":1}
I20260808 08:28:42.371620 19574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:42.371220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.194:38793)
0808 08:28:42.371458 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:42.371514 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":66,"threads_started":1}
I20260808 08:28:42.558264 19575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:42.557822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.195:48047)
0808 08:28:42.558108 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:42.558166 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":89,"threads_started":1}
I20260808 08:28:43.015405 19576 raft_consensus.cc:482] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 08:28:43.015636 19576 raft_consensus.cc:504] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "622af630070e4d129fdbd0e90b1cfd84" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34613 } } peers { permanent_uuid: "23f05ff8a1b04bb0a5a11717faec4639" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39403 } } peers { permanent_uuid: "5e6519a7db6245f9b459ee5905af464b" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45483 } }
I20260808 08:28:43.016821 19576 leader_election.cc:288] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 622af630070e4d129fdbd0e90b1cfd84 (127.22.235.195:34613), 23f05ff8a1b04bb0a5a11717faec4639 (127.22.235.194:39403)
Aug 08 08:28:43 dist-test-slave-f7th krb5kdc[17045](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177721, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for kudu/127.22.235.195@KRBTEST.COM
Aug 08 08:28:43 dist-test-slave-f7th krb5kdc[17045](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177721, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for kudu/127.22.235.194@KRBTEST.COM
I20260808 08:28:43.030959 19577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:43.017069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.195:34613 (local address 127.22.235.193:36123)
0808 08:28:43.017327 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:43.017344 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 08:28:43.017467 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:28:43.019259 (+ 1792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:28:43.019262 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:28:43.019814 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:28:43.020321 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:28:43.020335 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:28:43.021809 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:28:43.021811 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:28:43.022721 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:28:43.022731 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:28:43.023400 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:28:43.023403 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:28:43.023599 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:28:43.024412 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:28:43.024442 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:28:43.026045 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:28:43.028220 (+ 2175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:28:43.028224 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:28:43.028235 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:28:43.028588 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:28:43.028952 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:28:43.028955 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:28:43.028957 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:28:43.029141 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:28:43.029850 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:28:43.029857 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:28:43.030210 (+ 353us) client_negotiation.cc:769] Sending connection context
0808 08:28:43.030425 (+ 215us) client_negotiation.cc:240] Negotiation successful
0808 08:28:43.030818 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":278,"spinlock_wait_cycles":23680,"thread_start_us":80,"threads_started":1}
I20260808 08:28:43.031028 19583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:43.017640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:57137 (local address 127.22.235.194:39403)
0808 08:28:43.017931 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0808 08:28:43.017943 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 08:28:43.017976 (+ 33us) server_negotiation.cc:407] Connection header received
0808 08:28:43.018596 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:28:43.018601 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:28:43.018898 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:28:43.019178 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:28:43.020476 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:43.021676 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:43.022830 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:43.023264 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:43.023413 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:28:43.026581 (+ 3168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:28:43.026587 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:28:43.026601 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:28:43.026629 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:28:43.028311 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:28:43.028779 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:28:43.028784 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:28:43.028788 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:28:43.028903 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:28:43.029239 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:28:43.029242 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:28:43.029244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:28:43.029739 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:28:43.029873 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 08:28:43.030564 (+ 691us) server_negotiation.cc:299] Negotiation successful
0808 08:28:43.030839 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":96,"threads_started":1}
I20260808 08:28:43.031028 19581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:43.017229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:36123 (local address 127.22.235.195:34613)
0808 08:28:43.017737 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0808 08:28:43.017767 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0808 08:28:43.017815 (+ 48us) server_negotiation.cc:407] Connection header received
0808 08:28:43.018547 (+ 732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:28:43.018553 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:28:43.018898 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:28:43.019178 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:28:43.020488 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:43.021681 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:43.022839 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:43.023284 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:43.023435 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:28:43.026253 (+ 2818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:28:43.026258 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:28:43.026269 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:28:43.026296 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:28:43.028102 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:28:43.028707 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:28:43.028712 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:28:43.028716 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:28:43.028863 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:28:43.029237 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:28:43.029240 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:28:43.029241 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:28:43.029740 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:28:43.029873 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 08:28:43.030556 (+ 683us) server_negotiation.cc:299] Negotiation successful
0808 08:28:43.030839 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":93,"threads_started":1}
I20260808 08:28:43.030961 19582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:43.017405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.194:39403 (local address 127.22.235.193:57137)
0808 08:28:43.017628 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:43.017639 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 08:28:43.017723 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:28:43.019252 (+ 1529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:28:43.019258 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:28:43.019813 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:28:43.020320 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:28:43.020335 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:28:43.021800 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:28:43.021803 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:28:43.022722 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:28:43.022731 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:28:43.023375 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:28:43.023379 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:28:43.023597 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:28:43.024447 (+ 850us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:28:43.024459 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:28:43.026378 (+ 1919us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:28:43.028409 (+ 2031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:28:43.028413 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:28:43.028415 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:28:43.028692 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:28:43.028992 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:28:43.028995 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:28:43.028997 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:28:43.029141 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:28:43.029850 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:28:43.029857 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:28:43.030210 (+ 353us) client_negotiation.cc:769] Sending connection context
0808 08:28:43.030425 (+ 215us) client_negotiation.cc:240] Negotiation successful
0808 08:28:43.030753 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":205,"spinlock_wait_cycles":36608,"thread_start_us":65,"threads_started":1}
W20260808 08:28:43.033802 18964 leader_election.cc:341] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 23f05ff8a1b04bb0a5a11717faec4639 (127.22.235.194:39403): Not found: Tablet not found: e06544d83287436ca3694d3ce8f55658
W20260808 08:28:43.034010 18964 leader_election.cc:341] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 622af630070e4d129fdbd0e90b1cfd84 (127.22.235.195:34613): Not found: Tablet not found: e06544d83287436ca3694d3ce8f55658
I20260808 08:28:43.034070 18964 leader_election.cc:302] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e6519a7db6245f9b459ee5905af464b; no voters: 23f05ff8a1b04bb0a5a11717faec4639, 622af630070e4d129fdbd0e90b1cfd84
I20260808 08:28:43.034319 19576 raft_consensus.cc:2726] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 08:28:43.232728 19582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:43.232419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.193:42469)
0808 08:28:43.232551 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:43.232609 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 08:28:43.372845 19584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:43.372397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.194:48507)
0808 08:28:43.372679 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:43.372734 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":98,"threads_started":1}
I20260808 08:28:43.559636 19585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:43.559216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.195:58973)
0808 08:28:43.559494 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:43.559548 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":108,"threads_started":1}
I20260808 08:28:44.233829 19587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:44.233403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.193:49139)
0808 08:28:44.233676 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:44.233731 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":81,"threads_started":1}
I20260808 08:28:44.373878 19590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:44.373504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.194:38453)
0808 08:28:44.373721 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:44.373784 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":71,"threads_started":1}
I20260808 08:28:44.560874 19592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:44.560427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.195:49721)
0808 08:28:44.560731 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:44.560783 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":115,"threads_started":1}
I20260808 08:28:45.209754 19593 raft_consensus.cc:482] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 08:28:45.209924 19593 raft_consensus.cc:504] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "622af630070e4d129fdbd0e90b1cfd84" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34613 } } peers { permanent_uuid: "23f05ff8a1b04bb0a5a11717faec4639" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39403 } } peers { permanent_uuid: "5e6519a7db6245f9b459ee5905af464b" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45483 } }
I20260808 08:28:45.210366 19593 leader_election.cc:288] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 622af630070e4d129fdbd0e90b1cfd84 (127.22.235.195:34613), 23f05ff8a1b04bb0a5a11717faec4639 (127.22.235.194:39403)
W20260808 08:28:45.211323 18964 leader_election.cc:341] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 622af630070e4d129fdbd0e90b1cfd84 (127.22.235.195:34613): Not found: Tablet not found: e06544d83287436ca3694d3ce8f55658
W20260808 08:28:45.211498 18964 leader_election.cc:341] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 23f05ff8a1b04bb0a5a11717faec4639 (127.22.235.194:39403): Not found: Tablet not found: e06544d83287436ca3694d3ce8f55658
I20260808 08:28:45.211560 18964 leader_election.cc:302] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e6519a7db6245f9b459ee5905af464b; no voters: 23f05ff8a1b04bb0a5a11717faec4639, 622af630070e4d129fdbd0e90b1cfd84
I20260808 08:28:45.211732 19593 raft_consensus.cc:2726] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 08:28:45.235037 19594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:45.234585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.193:47865)
0808 08:28:45.234864 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:45.234926 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":101,"threads_started":1}
I20260808 08:28:45.374979 19595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:45.374585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.194:51137)
0808 08:28:45.374815 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:45.374868 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":75,"threads_started":1}
I20260808 08:28:45.561954 19597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:45.561546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.195:49997)
0808 08:28:45.561810 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:45.561863 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":92,"threads_started":1}
I20260808 08:28:46.236384 19598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:46.235920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.193:59733)
0808 08:28:46.236232 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:46.236284 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":87,"threads_started":1}
I20260808 08:28:46.376149 19599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:46.375699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.194:59097)
0808 08:28:46.376000 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:46.376054 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":89,"threads_started":1}
I20260808 08:28:46.563052 19600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:46.562633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.195:35473)
0808 08:28:46.562901 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:46.562959 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":75,"threads_started":1}
I20260808 08:28:46.932130 19601 raft_consensus.cc:482] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 08:28:46.932255 19601 raft_consensus.cc:504] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "622af630070e4d129fdbd0e90b1cfd84" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34613 } } peers { permanent_uuid: "23f05ff8a1b04bb0a5a11717faec4639" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39403 } } peers { permanent_uuid: "5e6519a7db6245f9b459ee5905af464b" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45483 } }
I20260808 08:28:46.932592 19601 leader_election.cc:288] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 622af630070e4d129fdbd0e90b1cfd84 (127.22.235.195:34613), 23f05ff8a1b04bb0a5a11717faec4639 (127.22.235.194:39403)
W20260808 08:28:46.933341 18964 leader_election.cc:341] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 622af630070e4d129fdbd0e90b1cfd84 (127.22.235.195:34613): Not found: Tablet not found: e06544d83287436ca3694d3ce8f55658
W20260808 08:28:46.933594 18964 leader_election.cc:341] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 23f05ff8a1b04bb0a5a11717faec4639 (127.22.235.194:39403): Not found: Tablet not found: e06544d83287436ca3694d3ce8f55658
I20260808 08:28:46.933665 18964 leader_election.cc:302] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e6519a7db6245f9b459ee5905af464b; no voters: 23f05ff8a1b04bb0a5a11717faec4639, 622af630070e4d129fdbd0e90b1cfd84
I20260808 08:28:46.933822 19601 raft_consensus.cc:2726] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 08:28:47.237521 19602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:47.237117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.193:59367)
0808 08:28:47.237360 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:47.237415 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":78,"threads_started":1}
I20260808 08:28:47.377296 19604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:47.376898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.194:50513)
0808 08:28:47.377148 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:47.377203 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":96,"threads_started":1}
I20260808 08:28:47.564143 19605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:47.563739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.195:58313)
0808 08:28:47.563999 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:47.564053 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":75,"threads_started":1}
I20260808 08:28:48.238690 19607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:48.238277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.193:39605)
0808 08:28:48.238532 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:48.238590 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":81,"threads_started":1}
I20260808 08:28:48.378530 19608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:48.378005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.194:43017)
0808 08:28:48.378328 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:48.378386 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":72,"threads_started":1}
I20260808 08:28:48.565304 19609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:48.564862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.195:37507)
0808 08:28:48.565157 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:48.565213 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":90,"threads_started":1}
I20260808 08:28:49.239770 19610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:49.239376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.193:42041)
0808 08:28:49.239615 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:49.239670 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":80,"threads_started":1}
I20260808 08:28:49.379688 19611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:49.379257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.194:56397)
0808 08:28:49.379538 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:49.379591 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":117,"threads_started":1}
I20260808 08:28:49.566381 19612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:49.565987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.195:42049)
0808 08:28:49.566237 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:49.566288 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":98,"threads_started":1}
I20260808 08:28:50.240895 19614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:50.240479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.193:60489)
0808 08:28:50.240745 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:50.240798 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":82,"threads_started":1}
I20260808 08:28:50.380897 19616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:50.380448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.194:54249)
0808 08:28:50.380719 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:50.380777 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":81,"threads_started":1}
I20260808 08:28:50.567484 19617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:50.567087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.195:42925)
0808 08:28:50.567338 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:50.567393 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":81,"threads_started":1}
I20260808 08:28:51.242203 19618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:51.241719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.193:59639)
0808 08:28:51.242038 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:51.242103 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":78,"threads_started":1}
I20260808 08:28:51.382033 19619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:51.381635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.194:48907)
0808 08:28:51.381876 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:51.381930 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":77,"threads_started":1}
I20260808 08:28:51.414239 19623 raft_consensus.cc:482] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 08:28:51.414398 19623 raft_consensus.cc:504] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "622af630070e4d129fdbd0e90b1cfd84" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34613 } } peers { permanent_uuid: "23f05ff8a1b04bb0a5a11717faec4639" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39403 } } peers { permanent_uuid: "5e6519a7db6245f9b459ee5905af464b" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45483 } }
I20260808 08:28:51.414784 19623 leader_election.cc:288] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 622af630070e4d129fdbd0e90b1cfd84 (127.22.235.195:34613), 23f05ff8a1b04bb0a5a11717faec4639 (127.22.235.194:39403)
W20260808 08:28:51.415580 18964 leader_election.cc:341] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 622af630070e4d129fdbd0e90b1cfd84 (127.22.235.195:34613): Not found: Tablet not found: e06544d83287436ca3694d3ce8f55658
W20260808 08:28:51.415697 18964 leader_election.cc:341] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 23f05ff8a1b04bb0a5a11717faec4639 (127.22.235.194:39403): Not found: Tablet not found: e06544d83287436ca3694d3ce8f55658
I20260808 08:28:51.415748 18964 leader_election.cc:302] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e6519a7db6245f9b459ee5905af464b; no voters: 23f05ff8a1b04bb0a5a11717faec4639, 622af630070e4d129fdbd0e90b1cfd84
I20260808 08:28:51.415913 19623 raft_consensus.cc:2726] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 08:28:51.568735 19624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:51.568246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.195:47761)
0808 08:28:51.568539 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:51.568606 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":78,"threads_started":1}
I20260808 08:28:52.243319 19625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:52.242908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.193:57423)
0808 08:28:52.243166 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:52.243222 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":85,"threads_started":1}
I20260808 08:28:52.383139 19626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:52.382718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.194:48591)
0808 08:28:52.382984 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:52.383041 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":86,"threads_started":1}
I20260808 08:28:52.569958 19628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:52.569524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.195:36765)
0808 08:28:52.569803 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:52.569860 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":103,"threads_started":1}
I20260808 08:28:53.244452 19629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:53.244045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.193:51109)
0808 08:28:53.244298 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:53.244353 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":78,"threads_started":1}
I20260808 08:28:53.384275 19630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:53.383878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.194:51441)
0808 08:28:53.384125 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:53.384180 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":86,"threads_started":1}
I20260808 08:28:53.571064 19633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:53.570685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.195:33973)
0808 08:28:53.570916 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:53.570968 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":75,"threads_started":1}
I20260808 08:28:54.245829 19634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:54.245387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.193:59545)
0808 08:28:54.245654 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:54.245720 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":83,"threads_started":1}
I20260808 08:28:54.385594 19635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:54.385117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.194:32829)
0808 08:28:54.385409 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:54.385475 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":92,"threads_started":1}
I20260808 08:28:54.572345 19636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:54.571864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.195:54033)
0808 08:28:54.572173 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:54.572243 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":77,"threads_started":1}
I20260808 08:28:55.247113 19639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:55.246603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.193:58751)
0808 08:28:55.246908 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:55.246988 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":102,"threads_started":1}
I20260808 08:28:55.386838 19640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:55.386402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.194:47901)
0808 08:28:55.386659 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:55.386719 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":81,"threads_started":1}
I20260808 08:28:55.488284 19643 raft_consensus.cc:482] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 08:28:55.488433 19643 raft_consensus.cc:504] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "622af630070e4d129fdbd0e90b1cfd84" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34613 } } peers { permanent_uuid: "23f05ff8a1b04bb0a5a11717faec4639" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 39403 } } peers { permanent_uuid: "5e6519a7db6245f9b459ee5905af464b" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45483 } }
I20260808 08:28:55.488897 19643 leader_election.cc:288] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 622af630070e4d129fdbd0e90b1cfd84 (127.22.235.195:34613), 23f05ff8a1b04bb0a5a11717faec4639 (127.22.235.194:39403)
W20260808 08:28:55.489751 18964 leader_election.cc:341] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 622af630070e4d129fdbd0e90b1cfd84 (127.22.235.195:34613): Not found: Tablet not found: e06544d83287436ca3694d3ce8f55658
W20260808 08:28:55.489889 18964 leader_election.cc:341] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 23f05ff8a1b04bb0a5a11717faec4639 (127.22.235.194:39403): Not found: Tablet not found: e06544d83287436ca3694d3ce8f55658
I20260808 08:28:55.489944 18964 leader_election.cc:302] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e6519a7db6245f9b459ee5905af464b; no voters: 23f05ff8a1b04bb0a5a11717faec4639, 622af630070e4d129fdbd0e90b1cfd84
I20260808 08:28:55.490101 19643 raft_consensus.cc:2726] T e06544d83287436ca3694d3ce8f55658 P 5e6519a7db6245f9b459ee5905af464b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 08:28:55.573485 19644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:55.573068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.195:52281)
0808 08:28:55.573313 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:55.573365 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":90,"threads_started":1}
I20260808 08:28:56.248512 19647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:56.248096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.193:43605)
0808 08:28:56.248353 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:56.248409 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":80,"threads_started":1}
I20260808 08:28:56.388087 19648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:56.387657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.194:60703)
0808 08:28:56.387932 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:56.387991 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":94,"threads_started":1}
I20260808 08:28:56.563974 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 18871
I20260808 08:28:56.574899 19650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:56.574477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:46859 (local address 127.22.235.195:50289)
0808 08:28:56.574740 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:56.574799 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.254:46859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":81,"threads_started":1}
I20260808 08:28:56.576092 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 19155
I20260808 08:28:56.592877 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 19363
2026-08-08T08:28:56Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59471 ms)
[----------] 30 tests from SecurityITest (243386 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 08:28:56 dist-test-slave-f7th krb5kdc[19656](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 08:28:56 dist-test-slave-f7th krb5kdc[19656](info): set up 2 sockets
Aug 08 08:28:56 dist-test-slave-f7th krb5kdc[19656](info): commencing operation
krb5kdc: starting...
W20260808 08:28:57.777362 23471 mini_kdc.cc:121] Time spent starting KDC: real 1.157s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Aug 08 08:28:57 dist-test-slave-f7th krb5kdc[19656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177737, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T08:28:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:28:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.235.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
I20260808 08:28:57.897369 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:43205
--webserver_interface=127.22.235.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:44261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:43205
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:28:58.009361 19675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:28:58.009657 19675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:28:58.009699 19675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:28:58.013787 19675 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:28:58.013857 19675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:28:58.013875 19675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:28:58.013895 19675 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:28:58.013923 19675 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:28:58.019317 19675 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:44261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:43205
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:43205
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.235.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19675
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:28:58.020387 19675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:28:58.021463 19675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:28:58.030035 19681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:28:58.030117 19675 server_base.cc:1034] running on GCE node
W20260808 08:28:58.030000 19682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:28:58.030161 19684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:28:58.030807 19675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:28:58.031896 19675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:28:58.033056 19675 hybrid_clock.cc:648] HybridClock initialized: now 1786177738033029 us; error 36 us; skew 500 ppm
Aug 08 08:28:58 dist-test-slave-f7th krb5kdc[19656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177738, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:28:58.035832 19675 init.cc:377] Logged in from keytab as kudu/127.22.235.254@KRBTEST.COM (short username kudu)
I20260808 08:28:58.036712 19675 webserver.cc:460] Webserver started at http://127.22.235.254:32837/ using document root <none> and password file <none>
I20260808 08:28:58.037252 19675 fs_manager.cc:359] Metadata directory not provided
I20260808 08:28:58.037319 19675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:28:58.037544 19675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:28:58.039501 19675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "bb02c2522815451e8e3c3b133e19600a"
format_stamp: "Formatted at 2026-08-08 08:28:58 on dist-test-slave-f7th"
server_key: "2d4a0ce4388c10c47389019476da3e2b"
server_key_iv: "8af8b1dd9f6c179c7e73af83edd8d73a"
server_key_version: "encryptionkey@0"
I20260808 08:28:58.040015 19675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "bb02c2522815451e8e3c3b133e19600a"
format_stamp: "Formatted at 2026-08-08 08:28:58 on dist-test-slave-f7th"
server_key: "2d4a0ce4388c10c47389019476da3e2b"
server_key_iv: "8af8b1dd9f6c179c7e73af83edd8d73a"
server_key_version: "encryptionkey@0"
I20260808 08:28:58.043529 19675 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260808 08:28:58.045830 19698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:28:58.046973 19675 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 08:28:58.047087 19675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "bb02c2522815451e8e3c3b133e19600a"
format_stamp: "Formatted at 2026-08-08 08:28:58 on dist-test-slave-f7th"
server_key: "2d4a0ce4388c10c47389019476da3e2b"
server_key_iv: "8af8b1dd9f6c179c7e73af83edd8d73a"
server_key_version: "encryptionkey@0"
I20260808 08:28:58.047178 19675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:28:58.061228 19675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:28:58.064113 19675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:28:58.064281 19675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:28:58.071136 19675 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.254:43205
I20260808 08:28:58.071158 19785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.254:43205 every 8 connection(s)
I20260808 08:28:58.072086 19675 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:28:58.073063 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 19675
I20260808 08:28:58.073179 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:28:58.073432 23471 external_mini_cluster.cc:1428] Setting key 076026ce12a63aee59a32bbe5cf01401
I20260808 08:28:58.075169 19786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:28:58.080605 19786 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb02c2522815451e8e3c3b133e19600a: Bootstrap starting.
Aug 08 08:28:58 dist-test-slave-f7th krb5kdc[19656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177737, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:28:58.082989 19786 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb02c2522815451e8e3c3b133e19600a: Neither blocks nor log segments found. Creating new log.
I20260808 08:28:58.083676 19786 log.cc:824] T 00000000000000000000000000000000 P bb02c2522815451e8e3c3b133e19600a: Log is configured to *not* fsync() on all Append() calls
I20260808 08:28:58.085517 19786 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb02c2522815451e8e3c3b133e19600a: No bootstrap required, opened a new log
I20260808 08:28:58.086470 19789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:58.074765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36904 (local address 127.22.235.254:43205)
0808 08:28:58.075190 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0808 08:28:58.075204 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 08:28:58.075249 (+ 45us) server_negotiation.cc:407] Connection header received
0808 08:28:58.075893 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:28:58.075910 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:28:58.076242 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:28:58.076545 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:28:58.077356 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:58.078326 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:58.079161 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:58.079611 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:58.079762 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:28:58.082287 (+ 2525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:28:58.082292 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:28:58.082304 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:28:58.082331 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:28:58.083890 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:28:58.084290 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:28:58.084295 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:28:58.084299 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:28:58.084405 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:28:58.084674 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:28:58.084677 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:28:58.084679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:28:58.085044 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:28:58.085232 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0808 08:28:58.085515 (+ 283us) server_negotiation.cc:299] Negotiation successful
0808 08:28:58.085807 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":150,"threads_started":1}
I20260808 08:28:58.088595 19786 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb02c2522815451e8e3c3b133e19600a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb02c2522815451e8e3c3b133e19600a" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 43205 } }
I20260808 08:28:58.088876 19786 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb02c2522815451e8e3c3b133e19600a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:28:58.088975 19786 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb02c2522815451e8e3c3b133e19600a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb02c2522815451e8e3c3b133e19600a, State: Initialized, Role: FOLLOWER
I20260808 08:28:58.089408 19786 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb02c2522815451e8e3c3b133e19600a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb02c2522815451e8e3c3b133e19600a" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 43205 } }
I20260808 08:28:58.089550 19786 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb02c2522815451e8e3c3b133e19600a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:28:58.089617 19786 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb02c2522815451e8e3c3b133e19600a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:28:58.089715 19786 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb02c2522815451e8e3c3b133e19600a [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:28:58.090597 19786 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb02c2522815451e8e3c3b133e19600a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb02c2522815451e8e3c3b133e19600a" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 43205 } }
I20260808 08:28:58.090915 19786 leader_election.cc:302] T 00000000000000000000000000000000 P bb02c2522815451e8e3c3b133e19600a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb02c2522815451e8e3c3b133e19600a; no voters:
I20260808 08:28:58.091187 19786 leader_election.cc:288] T 00000000000000000000000000000000 P bb02c2522815451e8e3c3b133e19600a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:28:58.091331 19792 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb02c2522815451e8e3c3b133e19600a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:28:58.091567 19792 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb02c2522815451e8e3c3b133e19600a [term 1 LEADER]: Becoming Leader. State: Replica: bb02c2522815451e8e3c3b133e19600a, State: Running, Role: LEADER
I20260808 08:28:58.091961 19792 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb02c2522815451e8e3c3b133e19600a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb02c2522815451e8e3c3b133e19600a" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 43205 } }
I20260808 08:28:58.092312 19786 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb02c2522815451e8e3c3b133e19600a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:28:58.093765 19794 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb02c2522815451e8e3c3b133e19600a [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb02c2522815451e8e3c3b133e19600a. Latest consensus state: current_term: 1 leader_uuid: "bb02c2522815451e8e3c3b133e19600a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb02c2522815451e8e3c3b133e19600a" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 43205 } } }
I20260808 08:28:58.093966 19794 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb02c2522815451e8e3c3b133e19600a [sys.catalog]: This master's current role is: LEADER
I20260808 08:28:58.093823 19793 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb02c2522815451e8e3c3b133e19600a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb02c2522815451e8e3c3b133e19600a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb02c2522815451e8e3c3b133e19600a" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 43205 } } }
I20260808 08:28:58.094231 19793 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb02c2522815451e8e3c3b133e19600a [sys.catalog]: This master's current role is: LEADER
W20260808 08:28:58.097124 19815 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bb02c2522815451e8e3c3b133e19600a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 08:28:58.097208 19815 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 08:28:58.097281 19813 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:28:58.098121 19813 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:28:58.103410 19813 catalog_manager.cc:1347] Generated new cluster ID: 77eab189683c4129add13c5c564f76ac
I20260808 08:28:58.103484 19813 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:28:58.114537 19813 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:28:58.115401 19813 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:28:58.120970 19813 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb02c2522815451e8e3c3b133e19600a: Generated new TSK 0
I20260808 08:28:58.121596 19813 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.235.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab.
Entry for principal kudu/127.22.235.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab.
I20260808 08:28:58.153599 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.193:0
--local_ip_for_outbound_sockets=127.22.235.193
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:43205
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab
--principal=kudu/127.22.235.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:44261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:28:58.265928 19823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:28:58.266194 19823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:28:58.266237 19823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:28:58.270641 19823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:28:58.270704 19823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:28:58.270764 19823 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.193
I20260808 08:28:58.276197 19823 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:44261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab
--principal=kudu/127.22.235.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:43205
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19823
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.193
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:28:58.277259 19823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:28:58.278219 19823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:28:58.287482 19837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:28:58.287495 19833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:28:58.287634 19823 server_base.cc:1034] running on GCE node
W20260808 08:28:58.287863 19832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:28:58.288235 19823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:28:58.288897 19823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:28:58.290057 19823 hybrid_clock.cc:648] HybridClock initialized: now 1786177738290040 us; error 31 us; skew 500 ppm
Aug 08 08:28:58 dist-test-slave-f7th krb5kdc[19656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177738, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:28:58.292698 19823 init.cc:377] Logged in from keytab as kudu/127.22.235.193@KRBTEST.COM (short username kudu)
I20260808 08:28:58.293604 19823 webserver.cc:460] Webserver started at http://127.22.235.193:33817/ using document root <none> and password file <none>
I20260808 08:28:58.294140 19823 fs_manager.cc:359] Metadata directory not provided
I20260808 08:28:58.294205 19823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:28:58.294400 19823 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:28:58.296362 19823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-0/data/instance:
uuid: "2eef3c9065584bbc80f07c36bf31ffb5"
format_stamp: "Formatted at 2026-08-08 08:28:58 on dist-test-slave-f7th"
server_key: "c1cec0a288423efcdf37bdad35db1dcf"
server_key_iv: "ec2afba710dcdfa4bb3fb110ed9a3110"
server_key_version: "encryptionkey@0"
I20260808 08:28:58.296904 19823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance:
uuid: "2eef3c9065584bbc80f07c36bf31ffb5"
format_stamp: "Formatted at 2026-08-08 08:28:58 on dist-test-slave-f7th"
server_key: "c1cec0a288423efcdf37bdad35db1dcf"
server_key_iv: "ec2afba710dcdfa4bb3fb110ed9a3110"
server_key_version: "encryptionkey@0"
I20260808 08:28:58.300557 19823 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 08:28:58.302861 19847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:28:58.303985 19823 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 08:28:58.304083 19823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-0/wal
uuid: "2eef3c9065584bbc80f07c36bf31ffb5"
format_stamp: "Formatted at 2026-08-08 08:28:58 on dist-test-slave-f7th"
server_key: "c1cec0a288423efcdf37bdad35db1dcf"
server_key_iv: "ec2afba710dcdfa4bb3fb110ed9a3110"
server_key_version: "encryptionkey@0"
I20260808 08:28:58.304174 19823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:28:58.316087 19823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:28:58.318737 19823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:28:58.318908 19823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:28:58.319507 19823 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:28:58.320340 19823 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:28:58.320389 19823 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:28:58.320434 19823 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:28:58.320464 19823 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:28:58.328723 19823 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.193:44755
I20260808 08:28:58.328748 20036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.193:44755 every 8 connection(s)
I20260808 08:28:58.329568 19823 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
Aug 08 08:28:58 dist-test-slave-f7th krb5kdc[19656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177738, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:28:58.339614 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 19823
I20260808 08:28:58.339745 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance
I20260808 08:28:58.340091 23471 external_mini_cluster.cc:1428] Setting key ebe4ea88a26814d6f51d97871ff137e5
I20260808 08:28:58.342535 19789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:58.331264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:32975 (local address 127.22.235.254:43205)
0808 08:28:58.331405 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 08:28:58.331409 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:28:58.332300 (+ 891us) server_negotiation.cc:407] Connection header received
0808 08:28:58.333622 (+ 1322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:28:58.333624 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:28:58.333673 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:28:58.333744 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:28:58.334867 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:58.335405 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:58.336460 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:58.336731 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:58.336807 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:28:58.339147 (+ 2340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:28:58.339150 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:28:58.339152 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:28:58.339176 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:28:58.340535 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:28:58.341049 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:28:58.341052 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:28:58.341054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:28:58.341138 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:28:58.341453 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:28:58.341457 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:28:58.341459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:28:58.341658 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:28:58.341734 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 08:28:58.342304 (+ 570us) server_negotiation.cc:299] Negotiation successful
0808 08:28:58.342404 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 08:28:58.343696 20040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:58.331657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:43205 (local address 127.22.235.193:32975)
0808 08:28:58.332156 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:58.332199 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 08:28:58.333432 (+ 1233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:28:58.333890 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:28:58.333897 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:28:58.334260 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:28:58.334726 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:28:58.334740 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:28:58.335494 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:28:58.335497 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:28:58.336367 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:28:58.336377 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:28:58.336846 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:28:58.336849 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:28:58.337006 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:28:58.337538 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:28:58.337554 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:28:58.338990 (+ 1436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:28:58.340666 (+ 1676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:28:58.340672 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:28:58.340687 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:28:58.340936 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:28:58.341226 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:28:58.341229 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:28:58.341231 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:28:58.341364 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:28:58.341761 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:28:58.341769 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:28:58.342085 (+ 316us) client_negotiation.cc:769] Sending connection context
0808 08:28:58.342287 (+ 202us) client_negotiation.cc:240] Negotiation successful
0808 08:28:58.342704 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":127,"threads_started":1}
I20260808 08:28:58.344928 20038 heartbeater.cc:347] Connected to a master server at 127.22.235.254:43205
I20260808 08:28:58.345158 20038 heartbeater.cc:464] Registering TS with master...
I20260808 08:28:58.345633 20038 heartbeater.cc:511] Master 127.22.235.254:43205 requested a full tablet report, sending...
I20260808 08:28:58.347126 19729 ts_manager.cc:194] Registered new tserver with Master: 2eef3c9065584bbc80f07c36bf31ffb5 (127.22.235.193:44755)
I20260808 08:28:58.348248 19729 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.193@KRBTEST.COM'} at 127.22.235.193:32975
WARNING: no policy specified for kudu/127.22.235.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab.
Entry for principal kudu/127.22.235.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab.
I20260808 08:28:58.365396 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.194:0
--local_ip_for_outbound_sockets=127.22.235.194
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:43205
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab
--principal=kudu/127.22.235.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:44261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:28:58.477097 20043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:28:58.477349 20043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:28:58.477397 20043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:28:58.481432 20043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:28:58.481494 20043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:28:58.481576 20043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.194
I20260808 08:28:58.487135 20043 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:44261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab
--principal=kudu/127.22.235.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:43205
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.194
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:28:58.488148 20043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:28:58.489225 20043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:28:58.498672 20064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:28:58.498741 20057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:28:58.498731 20060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:28:58.498940 20043 server_base.cc:1034] running on GCE node
I20260808 08:28:58.499317 20043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:28:58.499919 20043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:28:58.501089 20043 hybrid_clock.cc:648] HybridClock initialized: now 1786177738501062 us; error 39 us; skew 500 ppm
Aug 08 08:28:58 dist-test-slave-f7th krb5kdc[19656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177738, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:28:58.503624 20043 init.cc:377] Logged in from keytab as kudu/127.22.235.194@KRBTEST.COM (short username kudu)
I20260808 08:28:58.504475 20043 webserver.cc:460] Webserver started at http://127.22.235.194:46539/ using document root <none> and password file <none>
I20260808 08:28:58.505055 20043 fs_manager.cc:359] Metadata directory not provided
I20260808 08:28:58.505126 20043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:28:58.505342 20043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:28:58.507351 20043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-1/data/instance:
uuid: "b5c046f26d3f437ead6dfb3f820e8938"
format_stamp: "Formatted at 2026-08-08 08:28:58 on dist-test-slave-f7th"
server_key: "fa90e53183190b33a3d9ecb826596813"
server_key_iv: "f572770061ed6fe11b63d776cef3c624"
server_key_version: "encryptionkey@0"
I20260808 08:28:58.508023 20043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance:
uuid: "b5c046f26d3f437ead6dfb3f820e8938"
format_stamp: "Formatted at 2026-08-08 08:28:58 on dist-test-slave-f7th"
server_key: "fa90e53183190b33a3d9ecb826596813"
server_key_iv: "f572770061ed6fe11b63d776cef3c624"
server_key_version: "encryptionkey@0"
I20260808 08:28:58.511754 20043 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 08:28:58.514148 20075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:28:58.515316 20043 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 08:28:58.515426 20043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-1/wal
uuid: "b5c046f26d3f437ead6dfb3f820e8938"
format_stamp: "Formatted at 2026-08-08 08:28:58 on dist-test-slave-f7th"
server_key: "fa90e53183190b33a3d9ecb826596813"
server_key_iv: "f572770061ed6fe11b63d776cef3c624"
server_key_version: "encryptionkey@0"
I20260808 08:28:58.515523 20043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:28:58.534739 20043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:28:58.537441 20043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:28:58.537616 20043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:28:58.538199 20043 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:28:58.539024 20043 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:28:58.539073 20043 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:28:58.539125 20043 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:28:58.539155 20043 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:28:58.547596 20043 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.194:38831
I20260808 08:28:58.547623 20270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.194:38831 every 8 connection(s)
I20260808 08:28:58.548444 20043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
I20260808 08:28:58.551476 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 20043
I20260808 08:28:58.551635 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance
I20260808 08:28:58.551959 23471 external_mini_cluster.cc:1428] Setting key d0bacf1ba933211989f3c6920c734239
Aug 08 08:28:58 dist-test-slave-f7th krb5kdc[19656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177738, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.194@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:28:58.562489 19789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:58.550189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:53955 (local address 127.22.235.254:43205)
0808 08:28:58.550399 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 08:28:58.550404 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:28:58.551198 (+ 794us) server_negotiation.cc:407] Connection header received
0808 08:28:58.552833 (+ 1635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:28:58.552836 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:28:58.552892 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:28:58.552966 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:28:58.554405 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:58.554924 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:58.556229 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:58.556457 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:58.556553 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:28:58.558965 (+ 2412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:28:58.558970 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:28:58.558972 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:28:58.558997 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:28:58.560272 (+ 1275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:28:58.560894 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:28:58.560896 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:28:58.560898 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:28:58.560974 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:28:58.561323 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:28:58.561325 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:28:58.561326 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:28:58.561506 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:28:58.561577 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 08:28:58.562244 (+ 667us) server_negotiation.cc:299] Negotiation successful
0808 08:28:58.562351 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
WARNING: no policy specified for kudu/127.22.235.195@KRBTEST.COM; defaulting to no policy
I20260808 08:28:58.563385 20273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:58.550603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:43205 (local address 127.22.235.194:53955)
0808 08:28:58.551056 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:58.551096 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 08:28:58.552592 (+ 1496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:28:58.553151 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:28:58.553160 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:28:58.553614 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:28:58.554247 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:28:58.554265 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:28:58.555049 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:28:58.555052 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:28:58.556114 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:28:58.556124 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:28:58.556598 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:28:58.556602 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:28:58.556798 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:28:58.557311 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:28:58.557327 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:28:58.558758 (+ 1431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:28:58.560391 (+ 1633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:28:58.560399 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:28:58.560415 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:28:58.560796 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:28:58.561058 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:28:58.561062 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:28:58.561064 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:28:58.561241 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:28:58.561602 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:28:58.561609 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:28:58.561962 (+ 353us) client_negotiation.cc:769] Sending connection context
0808 08:28:58.562173 (+ 211us) client_negotiation.cc:240] Negotiation successful
0808 08:28:58.562513 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":127,"threads_started":1}
I20260808 08:28:58.564473 20271 heartbeater.cc:347] Connected to a master server at 127.22.235.254:43205
I20260808 08:28:58.564805 20271 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.195@KRBTEST.COM" created.
I20260808 08:28:58.565336 20271 heartbeater.cc:511] Master 127.22.235.254:43205 requested a full tablet report, sending...
I20260808 08:28:58.566402 19729 ts_manager.cc:194] Registered new tserver with Master: b5c046f26d3f437ead6dfb3f820e8938 (127.22.235.194:38831)
I20260808 08:28:58.566993 19729 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.194@KRBTEST.COM'} at 127.22.235.194:53955
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab.
Entry for principal kudu/127.22.235.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab.
I20260808 08:28:58.581444 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.195:0
--local_ip_for_outbound_sockets=127.22.235.195
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:43205
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab
--principal=kudu/127.22.235.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:44261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:28:58.695442 20278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:28:58.695684 20278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:28:58.695725 20278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:28:58.700081 20278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:28:58.700158 20278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:28:58.700238 20278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.195
I20260808 08:28:58.705785 20278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:44261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab
--principal=kudu/127.22.235.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:43205
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20278
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.195
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:28:58.707015 20278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:28:58.708124 20278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:28:58.717758 20290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:28:58.717769 20293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:28:58.717772 20286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:28:58.717931 20278 server_base.cc:1034] running on GCE node
I20260808 08:28:58.718417 20278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:28:58.719053 20278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:28:58.720209 20278 hybrid_clock.cc:648] HybridClock initialized: now 1786177738720185 us; error 41 us; skew 500 ppm
Aug 08 08:28:58 dist-test-slave-f7th krb5kdc[19656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177738, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:28:58.722867 20278 init.cc:377] Logged in from keytab as kudu/127.22.235.195@KRBTEST.COM (short username kudu)
I20260808 08:28:58.723702 20278 webserver.cc:460] Webserver started at http://127.22.235.195:42781/ using document root <none> and password file <none>
I20260808 08:28:58.724248 20278 fs_manager.cc:359] Metadata directory not provided
I20260808 08:28:58.724315 20278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:28:58.724522 20278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:28:58.726549 20278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-2/data/instance:
uuid: "2c25e584c1b54743987fad63d7a19911"
format_stamp: "Formatted at 2026-08-08 08:28:58 on dist-test-slave-f7th"
server_key: "7aecf36b745637a0ee980c2886ad0fc4"
server_key_iv: "e51cf22215d6935e6e439e740e611a0c"
server_key_version: "encryptionkey@0"
I20260808 08:28:58.727099 20278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance:
uuid: "2c25e584c1b54743987fad63d7a19911"
format_stamp: "Formatted at 2026-08-08 08:28:58 on dist-test-slave-f7th"
server_key: "7aecf36b745637a0ee980c2886ad0fc4"
server_key_iv: "e51cf22215d6935e6e439e740e611a0c"
server_key_version: "encryptionkey@0"
I20260808 08:28:58.730685 20278 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 08:28:58.732931 20305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:28:58.734067 20278 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:28:58.734175 20278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-2/wal
uuid: "2c25e584c1b54743987fad63d7a19911"
format_stamp: "Formatted at 2026-08-08 08:28:58 on dist-test-slave-f7th"
server_key: "7aecf36b745637a0ee980c2886ad0fc4"
server_key_iv: "e51cf22215d6935e6e439e740e611a0c"
server_key_version: "encryptionkey@0"
I20260808 08:28:58.734263 20278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:28:58.777266 20278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:28:58.780135 20278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:28:58.780323 20278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:28:58.780956 20278 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:28:58.781828 20278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:28:58.781878 20278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:28:58.781931 20278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:28:58.781967 20278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:28:58.790740 20278 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.195:33275
I20260808 08:28:58.790812 20491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.195:33275 every 8 connection(s)
I20260808 08:28:58.791662 20278 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
I20260808 08:28:58.798858 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 20278
I20260808 08:28:58.798956 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance
I20260808 08:28:58.799273 23471 external_mini_cluster.cc:1428] Setting key 50c6d9415e7c1d8ac4b22602ac8725ee
Aug 08 08:28:58 dist-test-slave-f7th krb5kdc[19656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177738, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.195@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:28:58.805645 19789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:58.793453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:38969 (local address 127.22.235.254:43205)
0808 08:28:58.793582 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 08:28:58.793587 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:28:58.794426 (+ 839us) server_negotiation.cc:407] Connection header received
0808 08:28:58.795949 (+ 1523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:28:58.795953 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:28:58.796014 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:28:58.796098 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:28:58.797584 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:58.798213 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:58.799537 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:58.799809 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:58.799912 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:28:58.802511 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:28:58.802515 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:28:58.802518 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:28:58.802545 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:28:58.803644 (+ 1099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:28:58.804116 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:28:58.804119 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:28:58.804121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:28:58.804204 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:28:58.804516 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:28:58.804519 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:28:58.804521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:28:58.804748 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:28:58.804839 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 08:28:58.805342 (+ 503us) server_negotiation.cc:299] Negotiation successful
0808 08:28:58.805482 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 08:28:58.806411 20494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:58.793816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:43205 (local address 127.22.235.195:38969)
0808 08:28:58.794258 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:58.794301 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 08:28:58.795707 (+ 1406us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:28:58.796273 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:28:58.796280 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:28:58.796737 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:28:58.797424 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:28:58.797443 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:28:58.798361 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:28:58.798365 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:28:58.799417 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:28:58.799427 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:28:58.799976 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:28:58.799981 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:28:58.800188 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:28:58.800819 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:28:58.800839 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:28:58.802324 (+ 1485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:28:58.803749 (+ 1425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:28:58.803755 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:28:58.803770 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:28:58.804024 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:28:58.804285 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:28:58.804288 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:28:58.804290 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:28:58.804424 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:28:58.804846 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:28:58.804852 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:28:58.805105 (+ 253us) client_negotiation.cc:769] Sending connection context
0808 08:28:58.805272 (+ 167us) client_negotiation.cc:240] Negotiation successful
0808 08:28:58.805602 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":118,"threads_started":1}
I20260808 08:28:58.807467 20492 heartbeater.cc:347] Connected to a master server at 127.22.235.254:43205
I20260808 08:28:58.807807 20492 heartbeater.cc:464] Registering TS with master...
I20260808 08:28:58.808326 20492 heartbeater.cc:511] Master 127.22.235.254:43205 requested a full tablet report, sending...
I20260808 08:28:58.809417 19729 ts_manager.cc:194] Registered new tserver with Master: 2c25e584c1b54743987fad63d7a19911 (127.22.235.195:33275)
I20260808 08:28:58.809975 19729 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.195@KRBTEST.COM'} at 127.22.235.195:38969
I20260808 08:28:58.813134 23471 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 08:28:58.821736 19789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:58.814978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36908 (local address 127.22.235.254:43205)
0808 08:28:58.815072 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0808 08:28:58.815076 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:28:58.815296 (+ 220us) server_negotiation.cc:407] Connection header received
0808 08:28:58.815400 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:28:58.815403 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:28:58.815445 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:28:58.815513 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:28:58.816175 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:58.816712 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:58.817512 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:58.817719 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:58.817805 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:28:58.819037 (+ 1232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:28:58.819041 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:28:58.819043 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:28:58.819065 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:28:58.820180 (+ 1115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:28:58.820557 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:28:58.820561 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:28:58.820562 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:28:58.820666 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:28:58.820914 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:28:58.820918 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:28:58.820919 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:28:58.821129 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:28:58.821248 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 08:28:58.821483 (+ 235us) server_negotiation.cc:299] Negotiation successful
0808 08:28:58.821593 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 08:28:58.824224 19729 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36908:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 08:28:58.827334 19729 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 08:28:58.844202 20509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:58.837664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:44755 (local address 127.0.0.1:50276)
0808 08:28:58.838843 (+ 1179us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:58.838899 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 08:28:58.839032 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:28:58.839861 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:28:58.839864 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:28:58.840032 (+ 168us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:28:58.840224 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:28:58.840232 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:28:58.841498 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:28:58.841501 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:28:58.842681 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:28:58.842692 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:28:58.843657 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:28:58.843660 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:28:58.843768 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:28:58.843814 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 08:28:58.843915 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 08:28:58.844039 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1072,"mutex_wait_us":114,"spinlock_wait_cycles":328576,"thread_start_us":109,"threads_started":1}
I20260808 08:28:58.844368 20511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:58.838377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.194:38831 (local address 127.0.0.1:44836)
0808 08:28:58.839156 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:58.839168 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 08:28:58.839990 (+ 822us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:28:58.840450 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:28:58.840453 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:28:58.840473 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:28:58.840559 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:28:58.840565 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:28:58.841911 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:28:58.841914 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:28:58.843058 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:28:58.843066 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:28:58.843892 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:28:58.843897 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:28:58.843998 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:28:58.844024 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 08:28:58.844091 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 08:28:58.844161 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":712,"mutex_wait_us":2,"thread_start_us":116,"threads_started":1}
I20260808 08:28:58.844653 20508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:58.837587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.195:33275 (local address 127.0.0.1:59346)
0808 08:28:58.838844 (+ 1257us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:58.838900 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 08:28:58.839072 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:28:58.839618 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:28:58.839624 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:28:58.840027 (+ 403us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:28:58.840241 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:28:58.840247 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:28:58.841766 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:28:58.841769 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:28:58.843008 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:28:58.843019 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:28:58.844222 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:28:58.844225 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:28:58.844325 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:28:58.844347 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 08:28:58.844416 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 08:28:58.844481 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1083,"mutex_wait_us":73,"spinlock_wait_cycles":816768,"thread_start_us":87,"threads_started":1}
I20260808 08:28:58.844826 20512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:58.838440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50276 (local address 127.22.235.193:44755)
0808 08:28:58.839461 (+ 1021us) server_negotiation.cc:206] Beginning negotiation
0808 08:28:58.839475 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 08:28:58.839493 (+ 18us) server_negotiation.cc:407] Connection header received
0808 08:28:58.839556 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:28:58.839561 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:28:58.839743 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:28:58.839865 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:28:58.840371 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:58.841263 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:58.842806 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:58.843529 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:58.843651 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:28:58.843789 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 08:28:58.844564 (+ 775us) server_negotiation.cc:299] Negotiation successful
0808 08:28:58.844689 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":937,"thread_start_us":76,"threads_started":1}
I20260808 08:28:58.845109 20510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:58.837750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59346 (local address 127.22.235.195:33275)
0808 08:28:58.838648 (+ 898us) server_negotiation.cc:206] Beginning negotiation
0808 08:28:58.838661 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 08:28:58.838994 (+ 333us) server_negotiation.cc:407] Connection header received
0808 08:28:58.839268 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:28:58.839275 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:28:58.839469 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:28:58.839593 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:28:58.840386 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:58.841614 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:58.843137 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:58.843935 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:58.844046 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:28:58.844160 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 08:28:58.844876 (+ 716us) server_negotiation.cc:299] Negotiation successful
0808 08:28:58.844976 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":786,"thread_start_us":281,"threads_started":1}
I20260808 08:28:58.845146 20513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:58.838563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44836 (local address 127.22.235.194:38831)
0808 08:28:58.838949 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0808 08:28:58.838971 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 08:28:58.839275 (+ 304us) server_negotiation.cc:407] Connection header received
0808 08:28:58.840138 (+ 863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:28:58.840143 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:28:58.840327 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:28:58.840472 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:28:58.840712 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:58.841780 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:58.843174 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:58.843780 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:58.843903 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:28:58.844037 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 08:28:58.844134 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 08:28:58.844226 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":128,"threads_started":1}
I20260808 08:28:58.847749 20158 tablet_service.cc:1467] Processing CreateTablet for tablet c5f64260d527400a96980dedddf9dd7c (DEFAULT_TABLE table=test-table [id=10dfa464277d465ebc03fd83300962e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:28:58.847846 19936 tablet_service.cc:1467] Processing CreateTablet for tablet c5f64260d527400a96980dedddf9dd7c (DEFAULT_TABLE table=test-table [id=10dfa464277d465ebc03fd83300962e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:28:58.847849 20378 tablet_service.cc:1467] Processing CreateTablet for tablet c5f64260d527400a96980dedddf9dd7c (DEFAULT_TABLE table=test-table [id=10dfa464277d465ebc03fd83300962e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:28:58.848953 20378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5f64260d527400a96980dedddf9dd7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:28:58.848953 19936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5f64260d527400a96980dedddf9dd7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:28:58.848961 20158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5f64260d527400a96980dedddf9dd7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:28:58.855572 20515 tablet_bootstrap.cc:492] T c5f64260d527400a96980dedddf9dd7c P b5c046f26d3f437ead6dfb3f820e8938: Bootstrap starting.
I20260808 08:28:58.856778 20516 tablet_bootstrap.cc:492] T c5f64260d527400a96980dedddf9dd7c P 2eef3c9065584bbc80f07c36bf31ffb5: Bootstrap starting.
I20260808 08:28:58.856976 20517 tablet_bootstrap.cc:492] T c5f64260d527400a96980dedddf9dd7c P 2c25e584c1b54743987fad63d7a19911: Bootstrap starting.
I20260808 08:28:58.858193 20515 tablet_bootstrap.cc:654] T c5f64260d527400a96980dedddf9dd7c P b5c046f26d3f437ead6dfb3f820e8938: Neither blocks nor log segments found. Creating new log.
I20260808 08:28:58.859021 20517 tablet_bootstrap.cc:654] T c5f64260d527400a96980dedddf9dd7c P 2c25e584c1b54743987fad63d7a19911: Neither blocks nor log segments found. Creating new log.
I20260808 08:28:58.859035 20516 tablet_bootstrap.cc:654] T c5f64260d527400a96980dedddf9dd7c P 2eef3c9065584bbc80f07c36bf31ffb5: Neither blocks nor log segments found. Creating new log.
I20260808 08:28:58.859611 20515 log.cc:824] T c5f64260d527400a96980dedddf9dd7c P b5c046f26d3f437ead6dfb3f820e8938: Log is configured to *not* fsync() on all Append() calls
I20260808 08:28:58.859865 20516 log.cc:824] T c5f64260d527400a96980dedddf9dd7c P 2eef3c9065584bbc80f07c36bf31ffb5: Log is configured to *not* fsync() on all Append() calls
I20260808 08:28:58.860246 20517 log.cc:824] T c5f64260d527400a96980dedddf9dd7c P 2c25e584c1b54743987fad63d7a19911: Log is configured to *not* fsync() on all Append() calls
I20260808 08:28:58.861570 20515 tablet_bootstrap.cc:492] T c5f64260d527400a96980dedddf9dd7c P b5c046f26d3f437ead6dfb3f820e8938: No bootstrap required, opened a new log
I20260808 08:28:58.861610 20516 tablet_bootstrap.cc:492] T c5f64260d527400a96980dedddf9dd7c P 2eef3c9065584bbc80f07c36bf31ffb5: No bootstrap required, opened a new log
I20260808 08:28:58.861754 20515 ts_tablet_manager.cc:1397] T c5f64260d527400a96980dedddf9dd7c P b5c046f26d3f437ead6dfb3f820e8938: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 08:28:58.861778 20516 ts_tablet_manager.cc:1397] T c5f64260d527400a96980dedddf9dd7c P 2eef3c9065584bbc80f07c36bf31ffb5: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 08:28:58.864534 20515 raft_consensus.cc:348] T c5f64260d527400a96980dedddf9dd7c P b5c046f26d3f437ead6dfb3f820e8938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c25e584c1b54743987fad63d7a19911" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 33275 } } peers { permanent_uuid: "2eef3c9065584bbc80f07c36bf31ffb5" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44755 } } peers { permanent_uuid: "b5c046f26d3f437ead6dfb3f820e8938" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 38831 } }
I20260808 08:28:58.864778 20515 raft_consensus.cc:374] T c5f64260d527400a96980dedddf9dd7c P b5c046f26d3f437ead6dfb3f820e8938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:28:58.864845 20515 raft_consensus.cc:729] T c5f64260d527400a96980dedddf9dd7c P b5c046f26d3f437ead6dfb3f820e8938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5c046f26d3f437ead6dfb3f820e8938, State: Initialized, Role: FOLLOWER
I20260808 08:28:58.864921 20516 raft_consensus.cc:348] T c5f64260d527400a96980dedddf9dd7c P 2eef3c9065584bbc80f07c36bf31ffb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c25e584c1b54743987fad63d7a19911" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 33275 } } peers { permanent_uuid: "2eef3c9065584bbc80f07c36bf31ffb5" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44755 } } peers { permanent_uuid: "b5c046f26d3f437ead6dfb3f820e8938" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 38831 } }
I20260808 08:28:58.865211 20516 raft_consensus.cc:374] T c5f64260d527400a96980dedddf9dd7c P 2eef3c9065584bbc80f07c36bf31ffb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:28:58.865265 20516 raft_consensus.cc:729] T c5f64260d527400a96980dedddf9dd7c P 2eef3c9065584bbc80f07c36bf31ffb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2eef3c9065584bbc80f07c36bf31ffb5, State: Initialized, Role: FOLLOWER
I20260808 08:28:58.865294 20515 consensus_queue.cc:260] T c5f64260d527400a96980dedddf9dd7c P b5c046f26d3f437ead6dfb3f820e8938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c25e584c1b54743987fad63d7a19911" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 33275 } } peers { permanent_uuid: "2eef3c9065584bbc80f07c36bf31ffb5" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44755 } } peers { permanent_uuid: "b5c046f26d3f437ead6dfb3f820e8938" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 38831 } }
I20260808 08:28:58.865706 20516 consensus_queue.cc:260] T c5f64260d527400a96980dedddf9dd7c P 2eef3c9065584bbc80f07c36bf31ffb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c25e584c1b54743987fad63d7a19911" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 33275 } } peers { permanent_uuid: "2eef3c9065584bbc80f07c36bf31ffb5" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44755 } } peers { permanent_uuid: "b5c046f26d3f437ead6dfb3f820e8938" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 38831 } }
I20260808 08:28:58.866302 20515 ts_tablet_manager.cc:1428] T c5f64260d527400a96980dedddf9dd7c P b5c046f26d3f437ead6dfb3f820e8938: Time spent starting tablet: real 0.004s user 0.002s sys 0.003s
I20260808 08:28:58.866374 20517 tablet_bootstrap.cc:492] T c5f64260d527400a96980dedddf9dd7c P 2c25e584c1b54743987fad63d7a19911: No bootstrap required, opened a new log
I20260808 08:28:58.866559 20517 ts_tablet_manager.cc:1397] T c5f64260d527400a96980dedddf9dd7c P 2c25e584c1b54743987fad63d7a19911: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20260808 08:28:58.866647 20516 ts_tablet_manager.cc:1428] T c5f64260d527400a96980dedddf9dd7c P 2eef3c9065584bbc80f07c36bf31ffb5: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 08:28:58.866971 20271 heartbeater.cc:503] Master 127.22.235.254:43205 was elected leader, sending a full tablet report...
I20260808 08:28:58.867002 20038 heartbeater.cc:503] Master 127.22.235.254:43205 was elected leader, sending a full tablet report...
I20260808 08:28:58.870272 20517 raft_consensus.cc:348] T c5f64260d527400a96980dedddf9dd7c P 2c25e584c1b54743987fad63d7a19911 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c25e584c1b54743987fad63d7a19911" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 33275 } } peers { permanent_uuid: "2eef3c9065584bbc80f07c36bf31ffb5" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44755 } } peers { permanent_uuid: "b5c046f26d3f437ead6dfb3f820e8938" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 38831 } }
I20260808 08:28:58.870532 20517 raft_consensus.cc:374] T c5f64260d527400a96980dedddf9dd7c P 2c25e584c1b54743987fad63d7a19911 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:28:58.870590 20517 raft_consensus.cc:729] T c5f64260d527400a96980dedddf9dd7c P 2c25e584c1b54743987fad63d7a19911 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c25e584c1b54743987fad63d7a19911, State: Initialized, Role: FOLLOWER
I20260808 08:28:58.871014 20517 consensus_queue.cc:260] T c5f64260d527400a96980dedddf9dd7c P 2c25e584c1b54743987fad63d7a19911 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c25e584c1b54743987fad63d7a19911" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 33275 } } peers { permanent_uuid: "2eef3c9065584bbc80f07c36bf31ffb5" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44755 } } peers { permanent_uuid: "b5c046f26d3f437ead6dfb3f820e8938" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 38831 } }
I20260808 08:28:58.871649 20492 heartbeater.cc:503] Master 127.22.235.254:43205 was elected leader, sending a full tablet report...
I20260808 08:28:58.871845 20517 ts_tablet_manager.cc:1428] T c5f64260d527400a96980dedddf9dd7c P 2c25e584c1b54743987fad63d7a19911: Time spent starting tablet: real 0.005s user 0.001s sys 0.005s
I20260808 08:28:58.973847 20523 raft_consensus.cc:482] T c5f64260d527400a96980dedddf9dd7c P b5c046f26d3f437ead6dfb3f820e8938 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 08:28:58.974062 20523 raft_consensus.cc:504] T c5f64260d527400a96980dedddf9dd7c P b5c046f26d3f437ead6dfb3f820e8938 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c25e584c1b54743987fad63d7a19911" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 33275 } } peers { permanent_uuid: "2eef3c9065584bbc80f07c36bf31ffb5" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44755 } } peers { permanent_uuid: "b5c046f26d3f437ead6dfb3f820e8938" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 38831 } }
I20260808 08:28:58.975273 20523 leader_election.cc:288] T c5f64260d527400a96980dedddf9dd7c P b5c046f26d3f437ead6dfb3f820e8938 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c25e584c1b54743987fad63d7a19911 (127.22.235.195:33275), 2eef3c9065584bbc80f07c36bf31ffb5 (127.22.235.193:44755)
I20260808 08:28:58.979936 20510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:58.975627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:36513 (local address 127.22.235.195:33275)
0808 08:28:58.975754 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 08:28:58.975759 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:28:58.975775 (+ 16us) server_negotiation.cc:407] Connection header received
0808 08:28:58.975929 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:28:58.975932 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:28:58.975983 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:28:58.976070 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:28:58.976335 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:58.977156 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:58.978745 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:58.979331 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:58.979452 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:28:58.979508 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 08:28:58.979741 (+ 233us) server_negotiation.cc:299] Negotiation successful
0808 08:28:58.979810 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 08:28:58.979938 20512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:58.975711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:37749 (local address 127.22.235.193:44755)
0808 08:28:58.975897 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 08:28:58.975901 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:28:58.976018 (+ 117us) server_negotiation.cc:407] Connection header received
0808 08:28:58.976104 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:28:58.976107 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:28:58.976155 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:28:58.976225 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:28:58.976476 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:58.977221 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:58.978735 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:58.979339 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:58.979452 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:28:58.979508 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 08:28:58.979740 (+ 232us) server_negotiation.cc:299] Negotiation successful
0808 08:28:58.979809 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260808 08:28:58.979945 20273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:58.975503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.195:33275 (local address 127.22.235.194:36513)
0808 08:28:58.975702 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:58.975716 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 08:28:58.975812 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:28:58.976092 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:28:58.976095 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:28:58.976111 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:28:58.976224 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:28:58.976231 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:28:58.977254 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:28:58.977256 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:28:58.978617 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:28:58.978630 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:28:58.979443 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:28:58.979447 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:28:58.979604 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:28:58.979644 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 08:28:58.979712 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 08:28:58.979796 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":103}
I20260808 08:28:58.979954 20530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:58.975573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:44755 (local address 127.22.235.194:37749)
0808 08:28:58.975937 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:58.975948 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 08:28:58.976026 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:28:58.976244 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:28:58.976247 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:28:58.976262 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:28:58.976372 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:28:58.976378 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:28:58.977311 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:28:58.977314 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:28:58.978620 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:28:58.978630 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:28:58.979443 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:28:58.979447 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:28:58.979607 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:28:58.979642 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 08:28:58.979716 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 08:28:58.979795 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"mutex_wait_us":83,"thread_start_us":134,"threads_started":1}
I20260808 08:28:58.980690 20410 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5f64260d527400a96980dedddf9dd7c" candidate_uuid: "b5c046f26d3f437ead6dfb3f820e8938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c25e584c1b54743987fad63d7a19911" is_pre_election: true
I20260808 08:28:58.980831 19964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5f64260d527400a96980dedddf9dd7c" candidate_uuid: "b5c046f26d3f437ead6dfb3f820e8938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2eef3c9065584bbc80f07c36bf31ffb5" is_pre_election: true
I20260808 08:28:58.980994 20410 raft_consensus.cc:2454] T c5f64260d527400a96980dedddf9dd7c P 2c25e584c1b54743987fad63d7a19911 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5c046f26d3f437ead6dfb3f820e8938 in term 0.
I20260808 08:28:58.981128 19964 raft_consensus.cc:2454] T c5f64260d527400a96980dedddf9dd7c P 2eef3c9065584bbc80f07c36bf31ffb5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5c046f26d3f437ead6dfb3f820e8938 in term 0.
I20260808 08:28:58.981470 20082 leader_election.cc:302] T c5f64260d527400a96980dedddf9dd7c P b5c046f26d3f437ead6dfb3f820e8938 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c25e584c1b54743987fad63d7a19911, b5c046f26d3f437ead6dfb3f820e8938; no voters:
I20260808 08:28:58.981738 20523 raft_consensus.cc:2781] T c5f64260d527400a96980dedddf9dd7c P b5c046f26d3f437ead6dfb3f820e8938 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 08:28:58.981823 20523 raft_consensus.cc:482] T c5f64260d527400a96980dedddf9dd7c P b5c046f26d3f437ead6dfb3f820e8938 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 08:28:58.981873 20523 raft_consensus.cc:3037] T c5f64260d527400a96980dedddf9dd7c P b5c046f26d3f437ead6dfb3f820e8938 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:28:58.982877 20523 raft_consensus.cc:504] T c5f64260d527400a96980dedddf9dd7c P b5c046f26d3f437ead6dfb3f820e8938 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c25e584c1b54743987fad63d7a19911" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 33275 } } peers { permanent_uuid: "2eef3c9065584bbc80f07c36bf31ffb5" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44755 } } peers { permanent_uuid: "b5c046f26d3f437ead6dfb3f820e8938" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 38831 } }
I20260808 08:28:58.983189 20523 leader_election.cc:288] T c5f64260d527400a96980dedddf9dd7c P b5c046f26d3f437ead6dfb3f820e8938 [CANDIDATE]: Term 1 election: Requested vote from peers 2c25e584c1b54743987fad63d7a19911 (127.22.235.195:33275), 2eef3c9065584bbc80f07c36bf31ffb5 (127.22.235.193:44755)
I20260808 08:28:58.983522 20410 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5f64260d527400a96980dedddf9dd7c" candidate_uuid: "b5c046f26d3f437ead6dfb3f820e8938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c25e584c1b54743987fad63d7a19911"
I20260808 08:28:58.983570 19964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5f64260d527400a96980dedddf9dd7c" candidate_uuid: "b5c046f26d3f437ead6dfb3f820e8938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2eef3c9065584bbc80f07c36bf31ffb5"
I20260808 08:28:58.983662 19964 raft_consensus.cc:3037] T c5f64260d527400a96980dedddf9dd7c P 2eef3c9065584bbc80f07c36bf31ffb5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:28:58.983654 20410 raft_consensus.cc:3037] T c5f64260d527400a96980dedddf9dd7c P 2c25e584c1b54743987fad63d7a19911 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:28:58.984952 19964 raft_consensus.cc:2454] T c5f64260d527400a96980dedddf9dd7c P 2eef3c9065584bbc80f07c36bf31ffb5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5c046f26d3f437ead6dfb3f820e8938 in term 1.
I20260808 08:28:58.984952 20410 raft_consensus.cc:2454] T c5f64260d527400a96980dedddf9dd7c P 2c25e584c1b54743987fad63d7a19911 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5c046f26d3f437ead6dfb3f820e8938 in term 1.
I20260808 08:28:58.985272 20082 leader_election.cc:302] T c5f64260d527400a96980dedddf9dd7c P b5c046f26d3f437ead6dfb3f820e8938 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2c25e584c1b54743987fad63d7a19911, 2eef3c9065584bbc80f07c36bf31ffb5, b5c046f26d3f437ead6dfb3f820e8938; no voters:
I20260808 08:28:58.985492 20523 raft_consensus.cc:2781] T c5f64260d527400a96980dedddf9dd7c P b5c046f26d3f437ead6dfb3f820e8938 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:28:58.985710 20523 raft_consensus.cc:686] T c5f64260d527400a96980dedddf9dd7c P b5c046f26d3f437ead6dfb3f820e8938 [term 1 LEADER]: Becoming Leader. State: Replica: b5c046f26d3f437ead6dfb3f820e8938, State: Running, Role: LEADER
I20260808 08:28:58.986073 20523 consensus_queue.cc:237] T c5f64260d527400a96980dedddf9dd7c P b5c046f26d3f437ead6dfb3f820e8938 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c25e584c1b54743987fad63d7a19911" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 33275 } } peers { permanent_uuid: "2eef3c9065584bbc80f07c36bf31ffb5" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44755 } } peers { permanent_uuid: "b5c046f26d3f437ead6dfb3f820e8938" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 38831 } }
I20260808 08:28:58.989414 19726 catalog_manager.cc:5495] T c5f64260d527400a96980dedddf9dd7c P b5c046f26d3f437ead6dfb3f820e8938 reported cstate change: term changed from 0 to 1, leader changed from <none> to b5c046f26d3f437ead6dfb3f820e8938 (127.22.235.194). New cstate: current_term: 1 leader_uuid: "b5c046f26d3f437ead6dfb3f820e8938" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c25e584c1b54743987fad63d7a19911" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 33275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2eef3c9065584bbc80f07c36bf31ffb5" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5c046f26d3f437ead6dfb3f820e8938" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 38831 } health_report { overall_health: HEALTHY } } }
I20260808 08:28:59.031533 20513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:59.028053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44852 (local address 127.22.235.194:38831)
0808 08:28:59.028189 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 08:28:59.028194 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:28:59.028209 (+ 15us) server_negotiation.cc:407] Connection header received
0808 08:28:59.028295 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:28:59.028297 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:28:59.028357 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:28:59.028453 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:28:59.028736 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:59.029242 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:59.030097 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:59.030310 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:59.030402 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:28:59.030705 (+ 303us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:28:59.031145 (+ 440us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 08:28:59.031258 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 08:28:59.031328 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 08:28:59.031388 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 08:28:59.037691 19964 raft_consensus.cc:1261] T c5f64260d527400a96980dedddf9dd7c P 2eef3c9065584bbc80f07c36bf31ffb5 [term 1 FOLLOWER]: Refusing update from remote peer b5c046f26d3f437ead6dfb3f820e8938: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:28:59.037691 20410 raft_consensus.cc:1261] T c5f64260d527400a96980dedddf9dd7c P 2c25e584c1b54743987fad63d7a19911 [term 1 FOLLOWER]: Refusing update from remote peer b5c046f26d3f437ead6dfb3f820e8938: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:28:59.038401 20523 consensus_queue.cc:1035] T c5f64260d527400a96980dedddf9dd7c P b5c046f26d3f437ead6dfb3f820e8938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c25e584c1b54743987fad63d7a19911" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 33275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:28:59.038599 20531 consensus_queue.cc:1035] T c5f64260d527400a96980dedddf9dd7c P b5c046f26d3f437ead6dfb3f820e8938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2eef3c9065584bbc80f07c36bf31ffb5" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 44755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 08:28:59.042730 20493 tablet.cc:2369] T c5f64260d527400a96980dedddf9dd7c P 2c25e584c1b54743987fad63d7a19911: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 08:28:59.045081 20535 mvcc.cc:205] Tried to move back new op lower bound from 7316184019090808832 to 7316184018889658368. Current Snapshot: MvccSnapshot[applied={T|T < 7316184019090808832}]
I20260808 08:28:59.045692 20537 mvcc.cc:205] Tried to move back new op lower bound from 7316184019090808832 to 7316184018889658368. Current Snapshot: MvccSnapshot[applied={T|T < 7316184019090808832}]
I20260808 08:28:59.048462 20539 mvcc.cc:205] Tried to move back new op lower bound from 7316184019090808832 to 7316184018889658368. Current Snapshot: MvccSnapshot[applied={T|T < 7316184019090808832}]
I20260808 08:28:59.053791 19729 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36908:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:28:59.053973 19729 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36908:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:28:59.056370 19729 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bb02c2522815451e8e3c3b133e19600a: Sending DeleteTablet for 3 replicas of tablet c5f64260d527400a96980dedddf9dd7c
I20260808 08:28:59.057132 19936 tablet_service.cc:1514] Processing DeleteTablet for tablet c5f64260d527400a96980dedddf9dd7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:28:59 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:50276
I20260808 08:28:59.057199 20378 tablet_service.cc:1514] Processing DeleteTablet for tablet c5f64260d527400a96980dedddf9dd7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:28:59 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:59346
I20260808 08:28:59.057312 20158 tablet_service.cc:1514] Processing DeleteTablet for tablet c5f64260d527400a96980dedddf9dd7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:28:59 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:44836
I20260808 08:28:59.057732 20556 tablet_replica.cc:330] T c5f64260d527400a96980dedddf9dd7c P b5c046f26d3f437ead6dfb3f820e8938: stopping tablet replica
I20260808 08:28:59.058008 20556 raft_consensus.cc:2229] T c5f64260d527400a96980dedddf9dd7c P b5c046f26d3f437ead6dfb3f820e8938 [term 1 LEADER]: Raft consensus shutting down.
I20260808 08:28:59.058180 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 19823
I20260808 08:28:59.058344 20556 raft_consensus.cc:2258] T c5f64260d527400a96980dedddf9dd7c P b5c046f26d3f437ead6dfb3f820e8938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:28:59.059796 20556 ts_tablet_manager.cc:1905] T c5f64260d527400a96980dedddf9dd7c P b5c046f26d3f437ead6dfb3f820e8938: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:28:59.068846 20554 tablet_replica.cc:330] T c5f64260d527400a96980dedddf9dd7c P 2c25e584c1b54743987fad63d7a19911: stopping tablet replica
I20260808 08:28:59.069214 20554 raft_consensus.cc:2229] T c5f64260d527400a96980dedddf9dd7c P 2c25e584c1b54743987fad63d7a19911 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 08:28:59.069566 20554 raft_consensus.cc:2258] T c5f64260d527400a96980dedddf9dd7c P 2c25e584c1b54743987fad63d7a19911 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:28:59.071112 20556 ts_tablet_manager.cc:1918] T c5f64260d527400a96980dedddf9dd7c P b5c046f26d3f437ead6dfb3f820e8938: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 08:28:59.071233 20556 log.cc:1196] T c5f64260d527400a96980dedddf9dd7c P b5c046f26d3f437ead6dfb3f820e8938: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-1/wal/wals/c5f64260d527400a96980dedddf9dd7c
I20260808 08:28:59.071548 20556 ts_tablet_manager.cc:1939] T c5f64260d527400a96980dedddf9dd7c P b5c046f26d3f437ead6dfb3f820e8938: Deleting consensus metadata
I20260808 08:28:59.072935 19706 catalog_manager.cc:4841] TS b5c046f26d3f437ead6dfb3f820e8938 (127.22.235.194:38831): tablet c5f64260d527400a96980dedddf9dd7c (table test-table [id=10dfa464277d465ebc03fd83300962e3]) successfully deleted
I20260808 08:28:59.074225 20554 ts_tablet_manager.cc:1905] T c5f64260d527400a96980dedddf9dd7c P 2c25e584c1b54743987fad63d7a19911: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:28:59.076984 20554 ts_tablet_manager.cc:1918] T c5f64260d527400a96980dedddf9dd7c P 2c25e584c1b54743987fad63d7a19911: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 08:28:59.077126 20554 log.cc:1196] T c5f64260d527400a96980dedddf9dd7c P 2c25e584c1b54743987fad63d7a19911: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786177493204855-23471-0/minicluster-data/ts-2/wal/wals/c5f64260d527400a96980dedddf9dd7c
I20260808 08:28:59.077440 20554 ts_tablet_manager.cc:1939] T c5f64260d527400a96980dedddf9dd7c P 2c25e584c1b54743987fad63d7a19911: Deleting consensus metadata
I20260808 08:28:59.078801 19706 catalog_manager.cc:4841] TS 2c25e584c1b54743987fad63d7a19911 (127.22.235.195:33275): tablet c5f64260d527400a96980dedddf9dd7c (table test-table [id=10dfa464277d465ebc03fd83300962e3]) successfully deleted
W20260808 08:28:59.078804 19699 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.235.193:44755 (error 108)
I20260808 08:28:59.079216 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 20043
I20260808 08:28:59.082165 20508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:59.081820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:44755 (local address 127.0.0.1:50282)
0808 08:28:59.082008 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:59.082076 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.193:44755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
W20260808 08:28:59.082490 19699 catalog_manager.cc:4584] TS 2eef3c9065584bbc80f07c36bf31ffb5 (127.22.235.193:44755): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c5f64260d527400a96980dedddf9dd7c: Network error: Client connection negotiation failed: client connection to 127.22.235.193:44755: connect: Connection refused (error 111)
I20260808 08:28:59.091106 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 20278
I20260808 08:28:59.123374 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 19675
2026-08-08T08:28:59Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2529 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-08T08:28:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:28:59Z Disabled control of system clock
I20260808 08:28:59.158439 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:41427
--webserver_interface=127.22.235.254
--webserver_port=0
--builtin_ntp_servers=127.22.235.212:44607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:41427
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 08:28:59.273052 20561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:28:59.273305 20561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:28:59.273352 20561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:28:59.277505 20561 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:28:59.277563 20561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:28:59.277578 20561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:28:59.277595 20561 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:28:59.277624 20561 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:28:59.282430 20561 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:44607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:41427
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:41427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.235.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:28:59.283548 20561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:28:59.284966 20561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:28:59.293156 20570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:28:59.293150 20573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:28:59.293197 20561 server_base.cc:1034] running on GCE node
W20260808 08:28:59.293150 20568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:28:59.293903 20561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:28:59.294961 20561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:28:59.296121 20561 hybrid_clock.cc:648] HybridClock initialized: now 1786177739296102 us; error 31 us; skew 500 ppm
I20260808 08:28:59.297879 20561 webserver.cc:460] Webserver started at http://127.22.235.254:43481/ using document root <none> and password file <none>
I20260808 08:28:59.298436 20561 fs_manager.cc:359] Metadata directory not provided
I20260808 08:28:59.298513 20561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:28:59.298724 20561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:28:59.301169 20561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "53514b1c95124e4185b238debf05523a"
format_stamp: "Formatted at 2026-08-08 08:28:59 on dist-test-slave-f7th"
server_key: "edee58f7770ceadd81960b0f6873c768"
server_key_iv: "b16ae3d8d03e66b7cb875ce3a0d96170"
server_key_version: "encryptionkey@0"
I20260808 08:28:59.301863 20561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "53514b1c95124e4185b238debf05523a"
format_stamp: "Formatted at 2026-08-08 08:28:59 on dist-test-slave-f7th"
server_key: "edee58f7770ceadd81960b0f6873c768"
server_key_iv: "b16ae3d8d03e66b7cb875ce3a0d96170"
server_key_version: "encryptionkey@0"
I20260808 08:28:59.306993 20561 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 08:28:59.311398 20581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:28:59.313040 20561 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 08:28:59.313174 20561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "53514b1c95124e4185b238debf05523a"
format_stamp: "Formatted at 2026-08-08 08:28:59 on dist-test-slave-f7th"
server_key: "edee58f7770ceadd81960b0f6873c768"
server_key_iv: "b16ae3d8d03e66b7cb875ce3a0d96170"
server_key_version: "encryptionkey@0"
I20260808 08:28:59.313299 20561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:28:59.329986 20561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:28:59.332393 20561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:28:59.332608 20561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:28:59.340674 20561 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.254:41427
I20260808 08:28:59.340685 20664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.254:41427 every 8 connection(s)
I20260808 08:28:59.341861 20561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:28:59.344254 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 20561
I20260808 08:28:59.344370 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:28:59.344686 23471 external_mini_cluster.cc:1428] Setting key c7c472dd5d26c0f7abbc21254259ed42
I20260808 08:28:59.346062 20665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:28:59.353845 20665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53514b1c95124e4185b238debf05523a: Bootstrap starting.
I20260808 08:28:59.354271 20668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:59.346154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40916 (local address 127.22.235.254:41427)
0808 08:28:59.346839 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0808 08:28:59.346855 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 08:28:59.346892 (+ 37us) server_negotiation.cc:407] Connection header received
0808 08:28:59.347570 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:28:59.347589 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:28:59.347977 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:28:59.348291 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:28:59.348785 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:59.349794 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:59.350669 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:59.351152 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:59.351300 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:28:59.352129 (+ 829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:28:59.352134 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:28:59.352148 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:28:59.352529 (+ 381us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:28:59.352585 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:28:59.352605 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:28:59.352723 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 08:28:59.353059 (+ 336us) server_negotiation.cc:299] Negotiation successful
0808 08:28:59.353338 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":129,"threads_started":1}
I20260808 08:28:59.356120 20665 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53514b1c95124e4185b238debf05523a: Neither blocks nor log segments found. Creating new log.
I20260808 08:28:59.356864 20665 log.cc:824] T 00000000000000000000000000000000 P 53514b1c95124e4185b238debf05523a: Log is configured to *not* fsync() on all Append() calls
I20260808 08:28:59.358646 20665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53514b1c95124e4185b238debf05523a: No bootstrap required, opened a new log
I20260808 08:28:59.361929 20665 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53514b1c95124e4185b238debf05523a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53514b1c95124e4185b238debf05523a" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 41427 } }
I20260808 08:28:59.362141 20665 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53514b1c95124e4185b238debf05523a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:28:59.362182 20665 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53514b1c95124e4185b238debf05523a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53514b1c95124e4185b238debf05523a, State: Initialized, Role: FOLLOWER
I20260808 08:28:59.362524 20665 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53514b1c95124e4185b238debf05523a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53514b1c95124e4185b238debf05523a" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 41427 } }
I20260808 08:28:59.362627 20665 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53514b1c95124e4185b238debf05523a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:28:59.362669 20665 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53514b1c95124e4185b238debf05523a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:28:59.362735 20665 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53514b1c95124e4185b238debf05523a [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:28:59.363518 20665 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53514b1c95124e4185b238debf05523a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53514b1c95124e4185b238debf05523a" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 41427 } }
I20260808 08:28:59.363808 20665 leader_election.cc:302] T 00000000000000000000000000000000 P 53514b1c95124e4185b238debf05523a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53514b1c95124e4185b238debf05523a; no voters:
I20260808 08:28:59.364099 20665 leader_election.cc:288] T 00000000000000000000000000000000 P 53514b1c95124e4185b238debf05523a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:28:59.364216 20672 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53514b1c95124e4185b238debf05523a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:28:59.364425 20672 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53514b1c95124e4185b238debf05523a [term 1 LEADER]: Becoming Leader. State: Replica: 53514b1c95124e4185b238debf05523a, State: Running, Role: LEADER
I20260808 08:28:59.364753 20672 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53514b1c95124e4185b238debf05523a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53514b1c95124e4185b238debf05523a" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 41427 } }
I20260808 08:28:59.365252 20665 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53514b1c95124e4185b238debf05523a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:28:59.366251 20675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53514b1c95124e4185b238debf05523a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53514b1c95124e4185b238debf05523a. Latest consensus state: current_term: 1 leader_uuid: "53514b1c95124e4185b238debf05523a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53514b1c95124e4185b238debf05523a" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 41427 } } }
I20260808 08:28:59.366376 20675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53514b1c95124e4185b238debf05523a [sys.catalog]: This master's current role is: LEADER
I20260808 08:28:59.366628 20673 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53514b1c95124e4185b238debf05523a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53514b1c95124e4185b238debf05523a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53514b1c95124e4185b238debf05523a" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 41427 } } }
I20260808 08:28:59.366719 20673 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53514b1c95124e4185b238debf05523a [sys.catalog]: This master's current role is: LEADER
I20260808 08:28:59.366851 20682 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:28:59.370111 20682 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:28:59.375510 20682 catalog_manager.cc:1347] Generated new cluster ID: 3d9d25154bba431297ea8bc2afb1a022
I20260808 08:28:59.375562 20682 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:28:59.402375 20682 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:28:59.403649 20682 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:28:59.415598 20682 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53514b1c95124e4185b238debf05523a: Generated new TSK 0
I20260808 08:28:59.416438 20682 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 08:28:59.430085 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.193:0
--local_ip_for_outbound_sockets=127.22.235.193
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:41427
--builtin_ntp_servers=127.22.235.212:44607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 08:28:59.576275 20696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:28:59.576536 20696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:28:59.576591 20696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:28:59.580569 20696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:28:59.580646 20696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:28:59.580724 20696 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.193
I20260808 08:28:59.585475 20696 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:44607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:41427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.193
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:28:59.586519 20696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:28:59.587898 20696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:28:59.597242 20704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:28:59.597206 20705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:28:59.597204 20710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:28:59.597544 20696 server_base.cc:1034] running on GCE node
I20260808 08:28:59.597880 20696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:28:59.598505 20696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:28:59.599660 20696 hybrid_clock.cc:648] HybridClock initialized: now 1786177739599645 us; error 28 us; skew 500 ppm
I20260808 08:28:59.601336 20696 webserver.cc:460] Webserver started at http://127.22.235.193:42885/ using document root <none> and password file <none>
I20260808 08:28:59.601866 20696 fs_manager.cc:359] Metadata directory not provided
I20260808 08:28:59.601922 20696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:28:59.602092 20696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:28:59.604013 20696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-0/data/instance:
uuid: "40467ea65a9d4147a12e7706278dbaf2"
format_stamp: "Formatted at 2026-08-08 08:28:59 on dist-test-slave-f7th"
server_key: "c96f65ab0e64a69d0477e398090dbc97"
server_key_iv: "17778662e1b6b7a7ac8c530f3a5fb044"
server_key_version: "encryptionkey@0"
I20260808 08:28:59.604485 20696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance:
uuid: "40467ea65a9d4147a12e7706278dbaf2"
format_stamp: "Formatted at 2026-08-08 08:28:59 on dist-test-slave-f7th"
server_key: "c96f65ab0e64a69d0477e398090dbc97"
server_key_iv: "17778662e1b6b7a7ac8c530f3a5fb044"
server_key_version: "encryptionkey@0"
I20260808 08:28:59.608101 20696 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 08:28:59.610553 20722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:28:59.611760 20696 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:28:59.611858 20696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-0/wal
uuid: "40467ea65a9d4147a12e7706278dbaf2"
format_stamp: "Formatted at 2026-08-08 08:28:59 on dist-test-slave-f7th"
server_key: "c96f65ab0e64a69d0477e398090dbc97"
server_key_iv: "17778662e1b6b7a7ac8c530f3a5fb044"
server_key_version: "encryptionkey@0"
I20260808 08:28:59.611941 20696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:28:59.637735 20696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:28:59.638458 20696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:28:59.638628 20696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:28:59.639210 20696 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:28:59.640183 20696 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:28:59.640231 20696 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:28:59.640273 20696 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:28:59.640295 20696 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:28:59.648672 20696 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.193:37223
I20260808 08:28:59.648698 20903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.193:37223 every 8 connection(s)
I20260808 08:28:59.649581 20696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
I20260808 08:28:59.651778 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 20696
I20260808 08:28:59.651876 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance
I20260808 08:28:59.652163 23471 external_mini_cluster.cc:1428] Setting key e3454f81244e8cb72e5dc9b2232796bd
I20260808 08:28:59.654321 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.194:0
--local_ip_for_outbound_sockets=127.22.235.194
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:41427
--builtin_ntp_servers=127.22.235.212:44607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 08:28:59.659860 20668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:59.651355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:47431 (local address 127.22.235.254:41427)
0808 08:28:59.651557 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 08:28:59.651562 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:28:59.652830 (+ 1268us) server_negotiation.cc:407] Connection header received
0808 08:28:59.654420 (+ 1590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:28:59.654424 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:28:59.654487 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:28:59.654573 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:28:59.655677 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:59.656200 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:59.657818 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:59.658064 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:59.658148 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:28:59.659355 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:28:59.659359 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:28:59.659361 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:28:59.659374 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:28:59.659400 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:28:59.659408 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:28:59.659467 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 08:28:59.659671 (+ 204us) server_negotiation.cc:299] Negotiation successful
0808 08:28:59.659746 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260808 08:28:59.661033 20906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:59.651816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:41427 (local address 127.22.235.193:47431)
0808 08:28:59.652645 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:59.652693 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 08:28:59.654195 (+ 1502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:28:59.654756 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:28:59.654765 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:28:59.654869 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:28:59.655529 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:28:59.655548 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:28:59.656324 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:28:59.656328 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:28:59.657681 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:28:59.657693 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:28:59.658219 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:28:59.658223 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:28:59.658412 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:28:59.659047 (+ 635us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 08:28:59.659049 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:28:59.659104 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 08:28:59.659185 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:28:59.659488 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:28:59.659496 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:28:59.659503 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 08:28:59.659635 (+ 132us) client_negotiation.cc:240] Negotiation successful
0808 08:28:59.659940 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":114,"threads_started":1}
I20260808 08:28:59.662209 20904 heartbeater.cc:347] Connected to a master server at 127.22.235.254:41427
I20260808 08:28:59.662478 20904 heartbeater.cc:464] Registering TS with master...
I20260808 08:28:59.663136 20904 heartbeater.cc:511] Master 127.22.235.254:41427 requested a full tablet report, sending...
I20260808 08:28:59.664803 20594 ts_manager.cc:194] Registered new tserver with Master: 40467ea65a9d4147a12e7706278dbaf2 (127.22.235.193:37223)
I20260808 08:28:59.666113 20594 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.235.193:47431
W20260808 08:28:59.768399 20908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:28:59.768700 20908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:28:59.768754 20908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:28:59.772671 20908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:28:59.772722 20908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:28:59.772785 20908 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.194
I20260808 08:28:59.777542 20908 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:44607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:41427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20908
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.194
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:28:59.778586 20908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:28:59.779963 20908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:28:59.789374 20917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:28:59.789456 20916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:28:59.789569 20908 server_base.cc:1034] running on GCE node
W20260808 08:28:59.789384 20927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:28:59.790124 20908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:28:59.790755 20908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:28:59.791920 20908 hybrid_clock.cc:648] HybridClock initialized: now 1786177739791901 us; error 29 us; skew 500 ppm
I20260808 08:28:59.793736 20908 webserver.cc:460] Webserver started at http://127.22.235.194:34037/ using document root <none> and password file <none>
I20260808 08:28:59.794301 20908 fs_manager.cc:359] Metadata directory not provided
I20260808 08:28:59.794368 20908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:28:59.794592 20908 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:28:59.796602 20908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-1/data/instance:
uuid: "11fd574efcfb4ccfb7e2ed56e3eed473"
format_stamp: "Formatted at 2026-08-08 08:28:59 on dist-test-slave-f7th"
server_key: "f2415c1d212683fa640e2d25d75a7650"
server_key_iv: "00cf25be7fcb3dea3ffea1200d108f2b"
server_key_version: "encryptionkey@0"
I20260808 08:28:59.797179 20908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance:
uuid: "11fd574efcfb4ccfb7e2ed56e3eed473"
format_stamp: "Formatted at 2026-08-08 08:28:59 on dist-test-slave-f7th"
server_key: "f2415c1d212683fa640e2d25d75a7650"
server_key_iv: "00cf25be7fcb3dea3ffea1200d108f2b"
server_key_version: "encryptionkey@0"
I20260808 08:28:59.801337 20908 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 08:28:59.803738 20934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:28:59.804963 20908 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:28:59.805073 20908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-1/wal
uuid: "11fd574efcfb4ccfb7e2ed56e3eed473"
format_stamp: "Formatted at 2026-08-08 08:28:59 on dist-test-slave-f7th"
server_key: "f2415c1d212683fa640e2d25d75a7650"
server_key_iv: "00cf25be7fcb3dea3ffea1200d108f2b"
server_key_version: "encryptionkey@0"
I20260808 08:28:59.805161 20908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:28:59.828236 20908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:28:59.828989 20908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:28:59.829181 20908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:28:59.829777 20908 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:28:59.830744 20908 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:28:59.830793 20908 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:28:59.830847 20908 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:28:59.830873 20908 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:28:59.839751 20908 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.194:40631
I20260808 08:28:59.839775 21111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.194:40631 every 8 connection(s)
I20260808 08:28:59.840668 20908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
I20260808 08:28:59.841723 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 20908
I20260808 08:28:59.841821 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance
I20260808 08:28:59.842144 23471 external_mini_cluster.cc:1428] Setting key d86b76370b0ca9d04e24070ffd705c7a
I20260808 08:28:59.844157 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.195:0
--local_ip_for_outbound_sockets=127.22.235.195
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:41427
--builtin_ntp_servers=127.22.235.212:44607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 08:28:59.850404 20668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:59.842523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:55147 (local address 127.22.235.254:41427)
0808 08:28:59.842679 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 08:28:59.842683 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:28:59.843893 (+ 1210us) server_negotiation.cc:407] Connection header received
0808 08:28:59.845380 (+ 1487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:28:59.845384 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:28:59.845441 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:28:59.845531 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:28:59.846435 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:59.847209 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:59.848317 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:28:59.848696 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:28:59.848807 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:28:59.849890 (+ 1083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:28:59.849895 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:28:59.849898 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:28:59.849916 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:28:59.849943 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:28:59.849954 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:28:59.850042 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 08:28:59.850165 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 08:28:59.850237 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 08:28:59.851265 21118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:28:59.842999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:41427 (local address 127.22.235.194:55147)
0808 08:28:59.843729 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0808 08:28:59.843781 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 08:28:59.845157 (+ 1376us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:28:59.845689 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:28:59.845696 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:28:59.845785 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:28:59.846293 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:28:59.846307 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:28:59.847316 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:28:59.847318 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:28:59.848191 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:28:59.848201 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:28:59.848841 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:28:59.848844 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:28:59.849004 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:28:59.849610 (+ 606us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 08:28:59.849612 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:28:59.849653 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 08:28:59.849720 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:28:59.850040 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:28:59.850046 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:28:59.850050 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 08:28:59.850144 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 08:28:59.850379 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":131,"threads_started":1}
I20260808 08:28:59.852442 21113 heartbeater.cc:347] Connected to a master server at 127.22.235.254:41427
I20260808 08:28:59.852712 21113 heartbeater.cc:464] Registering TS with master...
I20260808 08:28:59.853263 21113 heartbeater.cc:511] Master 127.22.235.254:41427 requested a full tablet report, sending...
I20260808 08:28:59.854312 20594 ts_manager.cc:194] Registered new tserver with Master: 11fd574efcfb4ccfb7e2ed56e3eed473 (127.22.235.194:40631)
I20260808 08:28:59.854889 20594 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.235.194:55147
W20260808 08:28:59.957041 21120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:28:59.957285 21120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:28:59.957321 21120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:28:59.961215 21120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:28:59.961262 21120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:28:59.961323 21120 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.195
I20260808 08:28:59.966317 21120 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:44607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:41427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21120
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.195
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:28:59.967427 21120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:28:59.968833 21120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:28:59.978022 21129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:28:59.978080 21128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:28:59.978080 21120 server_base.cc:1034] running on GCE node
W20260808 08:28:59.978048 21131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:28:59.978639 21120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:28:59.979249 21120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:28:59.980402 21120 hybrid_clock.cc:648] HybridClock initialized: now 1786177739980387 us; error 27 us; skew 500 ppm
I20260808 08:28:59.982146 21120 webserver.cc:460] Webserver started at http://127.22.235.195:40999/ using document root <none> and password file <none>
I20260808 08:28:59.982684 21120 fs_manager.cc:359] Metadata directory not provided
I20260808 08:28:59.982751 21120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:28:59.982957 21120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:28:59.985005 21120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-2/data/instance:
uuid: "24236a034d2b493d8ebb6efaa9ca7dda"
format_stamp: "Formatted at 2026-08-08 08:28:59 on dist-test-slave-f7th"
server_key: "8952d80eb2c425154c0f239090cb8834"
server_key_iv: "5fb11515fb3d56d0cd92399399b363eb"
server_key_version: "encryptionkey@0"
I20260808 08:28:59.985523 21120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance:
uuid: "24236a034d2b493d8ebb6efaa9ca7dda"
format_stamp: "Formatted at 2026-08-08 08:28:59 on dist-test-slave-f7th"
server_key: "8952d80eb2c425154c0f239090cb8834"
server_key_iv: "5fb11515fb3d56d0cd92399399b363eb"
server_key_version: "encryptionkey@0"
I20260808 08:28:59.989131 21120 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 08:28:59.991506 21139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:28:59.992645 21120 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:28:59.992753 21120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-2/wal
uuid: "24236a034d2b493d8ebb6efaa9ca7dda"
format_stamp: "Formatted at 2026-08-08 08:28:59 on dist-test-slave-f7th"
server_key: "8952d80eb2c425154c0f239090cb8834"
server_key_iv: "5fb11515fb3d56d0cd92399399b363eb"
server_key_version: "encryptionkey@0"
I20260808 08:28:59.992841 21120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:29:00.004869 21120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:29:00.005460 21120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:29:00.005607 21120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:29:00.006117 21120 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:29:00.007019 21120 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:29:00.007066 21120 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:00.007103 21120 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:29:00.007123 21120 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:00.015399 21120 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.195:37273
I20260808 08:29:00.015446 21336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.195:37273 every 8 connection(s)
I20260808 08:29:00.016280 21120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
I20260808 08:29:00.021006 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 21120
I20260808 08:29:00.021097 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance
I20260808 08:29:00.021322 23471 external_mini_cluster.cc:1428] Setting key a378f22498ee0f3f662509babae1a21e
I20260808 08:29:00.025435 20668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:00.018039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:34427 (local address 127.22.235.254:41427)
0808 08:29:00.018203 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:00.018208 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:00.019256 (+ 1048us) server_negotiation.cc:407] Connection header received
0808 08:29:00.020689 (+ 1433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:00.020693 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:00.020764 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:00.020847 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:00.021836 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:00.022544 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:00.023743 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:00.023956 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:00.024047 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:00.025068 (+ 1021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:00.025072 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:00.025074 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:29:00.025086 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:00.025105 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:29:00.025113 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:00.025169 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:00.025274 (+ 105us) server_negotiation.cc:299] Negotiation successful
0808 08:29:00.025323 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 08:29:00.026290 21343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:00.018471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:41427 (local address 127.22.235.195:34427)
0808 08:29:00.019117 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:00.019171 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:00.020404 (+ 1233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:00.021072 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:00.021082 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:00.021184 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:29:00.021689 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:00.021704 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:00.022667 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:00.022671 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:00.023632 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:00.023642 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:00.024089 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:00.024091 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:00.024231 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:00.024814 (+ 583us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 08:29:00.024815 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:29:00.024857 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 08:29:00.024928 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:29:00.025174 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:29:00.025179 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:29:00.025182 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 08:29:00.025267 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 08:29:00.025479 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":106,"threads_started":1}
I20260808 08:29:00.027362 21337 heartbeater.cc:347] Connected to a master server at 127.22.235.254:41427
I20260808 08:29:00.027575 21337 heartbeater.cc:464] Registering TS with master...
I20260808 08:29:00.028088 21337 heartbeater.cc:511] Master 127.22.235.254:41427 requested a full tablet report, sending...
I20260808 08:29:00.029072 20594 ts_manager.cc:194] Registered new tserver with Master: 24236a034d2b493d8ebb6efaa9ca7dda (127.22.235.195:37273)
I20260808 08:29:00.029623 20594 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.235.195:34427
I20260808 08:29:00.034875 23471 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 08:29:00.040006 20668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:00.036739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40932 (local address 127.22.235.254:41427)
0808 08:29:00.036870 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:00.036874 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:00.037054 (+ 180us) server_negotiation.cc:407] Connection header received
0808 08:29:00.037141 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:00.037144 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:00.037180 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:00.037240 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:00.037478 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:00.037947 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:00.038723 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:00.038917 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:00.038991 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:00.039680 (+ 689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:00.039683 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:00.039685 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:29:00.039697 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:00.039716 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:29:00.039724 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:00.039786 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:00.039836 (+ 50us) server_negotiation.cc:299] Negotiation successful
0808 08:29:00.039880 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 08:29:00.042557 20594 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40932:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 08:29:00.045490 20594 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 08:29:00.059969 21363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:00.053876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:37223 (local address 127.0.0.1:54526)
0808 08:29:00.054903 (+ 1027us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:00.054915 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:00.054999 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:00.056207 (+ 1208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:00.056210 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:00.056227 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:29:00.056312 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:00.056319 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:00.057340 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:00.057344 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:00.058515 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:00.058526 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:00.059411 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:00.059415 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:00.059529 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:00.059576 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 08:29:00.059690 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 08:29:00.059815 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":963,"mutex_wait_us":71,"thread_start_us":94,"threads_started":1}
I20260808 08:29:00.060004 21364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:00.054582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54526 (local address 127.22.235.193:37223)
0808 08:29:00.055853 (+ 1271us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:00.055864 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:00.055886 (+ 22us) server_negotiation.cc:407] Connection header received
0808 08:29:00.055955 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:00.055960 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:00.056094 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:00.056214 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:29:00.056415 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:00.057209 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:00.058640 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:00.059286 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:00.059406 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:00.059587 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:00.059727 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 08:29:00.059841 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1174,"thread_start_us":140,"threads_started":1}
I20260808 08:29:00.060178 21358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:00.053779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.194:40631 (local address 127.0.0.1:36434)
0808 08:29:00.054106 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:00.054163 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:00.054290 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:00.055698 (+ 1408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:00.055701 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:00.055720 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:29:00.055856 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:00.055863 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:00.057327 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:00.057330 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:00.058905 (+ 1575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:00.058917 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:00.059830 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:00.059833 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:00.059917 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:00.059939 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 08:29:00.059998 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 08:29:00.060062 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":225,"thread_start_us":62,"threads_started":1}
I20260808 08:29:00.060915 21361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:00.054584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.195:37273 (local address 127.0.0.1:53780)
0808 08:29:00.055075 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:00.055089 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:00.055180 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:00.055606 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:00.055612 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:00.055636 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:29:00.055802 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:00.055810 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:00.057327 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:00.057330 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:00.058905 (+ 1575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:00.058917 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:00.060516 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:00.060520 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:00.060609 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:00.060657 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 08:29:00.060718 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 08:29:00.060785 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"mutex_wait_us":113,"thread_start_us":93,"threads_started":1}
I20260808 08:29:00.060932 21362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:00.053921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36434 (local address 127.22.235.194:40631)
0808 08:29:00.055280 (+ 1359us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:00.055308 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:00.055329 (+ 21us) server_negotiation.cc:407] Connection header received
0808 08:29:00.055392 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:00.055395 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:00.055544 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:00.055707 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:29:00.055980 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:00.057190 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:00.059063 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:00.059706 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:00.060447 (+ 741us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:00.060640 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:00.060689 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 08:29:00.060777 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1251,"thread_start_us":145,"threads_started":1}
I20260808 08:29:00.061431 21365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:00.054744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53780 (local address 127.22.235.195:37273)
0808 08:29:00.055204 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:00.055213 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:00.055243 (+ 30us) server_negotiation.cc:407] Connection header received
0808 08:29:00.055303 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:00.055307 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:00.055457 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:00.055578 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:29:00.055937 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:00.057190 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:00.059043 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:00.059782 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:00.059894 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:00.061114 (+ 1220us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:00.061167 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 08:29:00.061265 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":105,"threads_started":1}
I20260808 08:29:00.063396 20803 tablet_service.cc:1467] Processing CreateTablet for tablet fca7001d55534ae8b1f488f402082e72 (DEFAULT_TABLE table=test-table [id=e570c86c4dd14d5db1d5a88268968cc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:29:00.063596 21016 tablet_service.cc:1467] Processing CreateTablet for tablet fca7001d55534ae8b1f488f402082e72 (DEFAULT_TABLE table=test-table [id=e570c86c4dd14d5db1d5a88268968cc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:29:00.063907 21230 tablet_service.cc:1467] Processing CreateTablet for tablet fca7001d55534ae8b1f488f402082e72 (DEFAULT_TABLE table=test-table [id=e570c86c4dd14d5db1d5a88268968cc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:29:00.064333 20803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fca7001d55534ae8b1f488f402082e72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:29:00.064664 21016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fca7001d55534ae8b1f488f402082e72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:29:00.064875 21230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fca7001d55534ae8b1f488f402082e72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:29:00.070775 21369 tablet_bootstrap.cc:492] T fca7001d55534ae8b1f488f402082e72 P 40467ea65a9d4147a12e7706278dbaf2: Bootstrap starting.
I20260808 08:29:00.072566 21370 tablet_bootstrap.cc:492] T fca7001d55534ae8b1f488f402082e72 P 11fd574efcfb4ccfb7e2ed56e3eed473: Bootstrap starting.
I20260808 08:29:00.072953 21372 tablet_bootstrap.cc:492] T fca7001d55534ae8b1f488f402082e72 P 24236a034d2b493d8ebb6efaa9ca7dda: Bootstrap starting.
I20260808 08:29:00.073057 21369 tablet_bootstrap.cc:654] T fca7001d55534ae8b1f488f402082e72 P 40467ea65a9d4147a12e7706278dbaf2: Neither blocks nor log segments found. Creating new log.
I20260808 08:29:00.073932 21369 log.cc:824] T fca7001d55534ae8b1f488f402082e72 P 40467ea65a9d4147a12e7706278dbaf2: Log is configured to *not* fsync() on all Append() calls
I20260808 08:29:00.074334 21370 tablet_bootstrap.cc:654] T fca7001d55534ae8b1f488f402082e72 P 11fd574efcfb4ccfb7e2ed56e3eed473: Neither blocks nor log segments found. Creating new log.
I20260808 08:29:00.075048 21372 tablet_bootstrap.cc:654] T fca7001d55534ae8b1f488f402082e72 P 24236a034d2b493d8ebb6efaa9ca7dda: Neither blocks nor log segments found. Creating new log.
I20260808 08:29:00.075171 21370 log.cc:824] T fca7001d55534ae8b1f488f402082e72 P 11fd574efcfb4ccfb7e2ed56e3eed473: Log is configured to *not* fsync() on all Append() calls
I20260808 08:29:00.075796 21372 log.cc:824] T fca7001d55534ae8b1f488f402082e72 P 24236a034d2b493d8ebb6efaa9ca7dda: Log is configured to *not* fsync() on all Append() calls
I20260808 08:29:00.076069 21369 tablet_bootstrap.cc:492] T fca7001d55534ae8b1f488f402082e72 P 40467ea65a9d4147a12e7706278dbaf2: No bootstrap required, opened a new log
I20260808 08:29:00.076270 21369 ts_tablet_manager.cc:1397] T fca7001d55534ae8b1f488f402082e72 P 40467ea65a9d4147a12e7706278dbaf2: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 08:29:00.077329 21370 tablet_bootstrap.cc:492] T fca7001d55534ae8b1f488f402082e72 P 11fd574efcfb4ccfb7e2ed56e3eed473: No bootstrap required, opened a new log
I20260808 08:29:00.077482 21372 tablet_bootstrap.cc:492] T fca7001d55534ae8b1f488f402082e72 P 24236a034d2b493d8ebb6efaa9ca7dda: No bootstrap required, opened a new log
I20260808 08:29:00.077509 21370 ts_tablet_manager.cc:1397] T fca7001d55534ae8b1f488f402082e72 P 11fd574efcfb4ccfb7e2ed56e3eed473: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260808 08:29:00.077662 21372 ts_tablet_manager.cc:1397] T fca7001d55534ae8b1f488f402082e72 P 24236a034d2b493d8ebb6efaa9ca7dda: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 08:29:00.080574 21372 raft_consensus.cc:348] T fca7001d55534ae8b1f488f402082e72 P 24236a034d2b493d8ebb6efaa9ca7dda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11fd574efcfb4ccfb7e2ed56e3eed473" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 40631 } } peers { permanent_uuid: "40467ea65a9d4147a12e7706278dbaf2" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 37223 } } peers { permanent_uuid: "24236a034d2b493d8ebb6efaa9ca7dda" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37273 } }
I20260808 08:29:00.080550 21369 raft_consensus.cc:348] T fca7001d55534ae8b1f488f402082e72 P 40467ea65a9d4147a12e7706278dbaf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11fd574efcfb4ccfb7e2ed56e3eed473" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 40631 } } peers { permanent_uuid: "40467ea65a9d4147a12e7706278dbaf2" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 37223 } } peers { permanent_uuid: "24236a034d2b493d8ebb6efaa9ca7dda" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37273 } }
I20260808 08:29:00.080828 21372 raft_consensus.cc:374] T fca7001d55534ae8b1f488f402082e72 P 24236a034d2b493d8ebb6efaa9ca7dda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:29:00.080832 21369 raft_consensus.cc:374] T fca7001d55534ae8b1f488f402082e72 P 40467ea65a9d4147a12e7706278dbaf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:29:00.080885 21372 raft_consensus.cc:729] T fca7001d55534ae8b1f488f402082e72 P 24236a034d2b493d8ebb6efaa9ca7dda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24236a034d2b493d8ebb6efaa9ca7dda, State: Initialized, Role: FOLLOWER
I20260808 08:29:00.080893 21369 raft_consensus.cc:729] T fca7001d55534ae8b1f488f402082e72 P 40467ea65a9d4147a12e7706278dbaf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40467ea65a9d4147a12e7706278dbaf2, State: Initialized, Role: FOLLOWER
I20260808 08:29:00.081332 21372 consensus_queue.cc:260] T fca7001d55534ae8b1f488f402082e72 P 24236a034d2b493d8ebb6efaa9ca7dda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11fd574efcfb4ccfb7e2ed56e3eed473" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 40631 } } peers { permanent_uuid: "40467ea65a9d4147a12e7706278dbaf2" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 37223 } } peers { permanent_uuid: "24236a034d2b493d8ebb6efaa9ca7dda" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37273 } }
I20260808 08:29:00.081344 21369 consensus_queue.cc:260] T fca7001d55534ae8b1f488f402082e72 P 40467ea65a9d4147a12e7706278dbaf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11fd574efcfb4ccfb7e2ed56e3eed473" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 40631 } } peers { permanent_uuid: "40467ea65a9d4147a12e7706278dbaf2" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 37223 } } peers { permanent_uuid: "24236a034d2b493d8ebb6efaa9ca7dda" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37273 } }
I20260808 08:29:00.081468 21370 raft_consensus.cc:348] T fca7001d55534ae8b1f488f402082e72 P 11fd574efcfb4ccfb7e2ed56e3eed473 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11fd574efcfb4ccfb7e2ed56e3eed473" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 40631 } } peers { permanent_uuid: "40467ea65a9d4147a12e7706278dbaf2" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 37223 } } peers { permanent_uuid: "24236a034d2b493d8ebb6efaa9ca7dda" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37273 } }
I20260808 08:29:00.081696 21370 raft_consensus.cc:374] T fca7001d55534ae8b1f488f402082e72 P 11fd574efcfb4ccfb7e2ed56e3eed473 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:29:00.081743 21370 raft_consensus.cc:729] T fca7001d55534ae8b1f488f402082e72 P 11fd574efcfb4ccfb7e2ed56e3eed473 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11fd574efcfb4ccfb7e2ed56e3eed473, State: Initialized, Role: FOLLOWER
I20260808 08:29:00.082154 21370 consensus_queue.cc:260] T fca7001d55534ae8b1f488f402082e72 P 11fd574efcfb4ccfb7e2ed56e3eed473 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11fd574efcfb4ccfb7e2ed56e3eed473" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 40631 } } peers { permanent_uuid: "40467ea65a9d4147a12e7706278dbaf2" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 37223 } } peers { permanent_uuid: "24236a034d2b493d8ebb6efaa9ca7dda" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37273 } }
I20260808 08:29:00.082324 21372 ts_tablet_manager.cc:1428] T fca7001d55534ae8b1f488f402082e72 P 24236a034d2b493d8ebb6efaa9ca7dda: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 08:29:00.082330 21369 ts_tablet_manager.cc:1428] T fca7001d55534ae8b1f488f402082e72 P 40467ea65a9d4147a12e7706278dbaf2: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 08:29:00.082677 20904 heartbeater.cc:503] Master 127.22.235.254:41427 was elected leader, sending a full tablet report...
I20260808 08:29:00.082700 21337 heartbeater.cc:503] Master 127.22.235.254:41427 was elected leader, sending a full tablet report...
I20260808 08:29:00.082805 21113 heartbeater.cc:503] Master 127.22.235.254:41427 was elected leader, sending a full tablet report...
I20260808 08:29:00.083009 21370 ts_tablet_manager.cc:1428] T fca7001d55534ae8b1f488f402082e72 P 11fd574efcfb4ccfb7e2ed56e3eed473: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
W20260808 08:29:00.091527 21116 tablet.cc:2369] T fca7001d55534ae8b1f488f402082e72 P 11fd574efcfb4ccfb7e2ed56e3eed473: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 08:29:00.150679 20905 tablet.cc:2369] T fca7001d55534ae8b1f488f402082e72 P 40467ea65a9d4147a12e7706278dbaf2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 08:29:00.155733 21379 raft_consensus.cc:482] T fca7001d55534ae8b1f488f402082e72 P 24236a034d2b493d8ebb6efaa9ca7dda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 08:29:00.155915 21379 raft_consensus.cc:504] T fca7001d55534ae8b1f488f402082e72 P 24236a034d2b493d8ebb6efaa9ca7dda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11fd574efcfb4ccfb7e2ed56e3eed473" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 40631 } } peers { permanent_uuid: "40467ea65a9d4147a12e7706278dbaf2" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 37223 } } peers { permanent_uuid: "24236a034d2b493d8ebb6efaa9ca7dda" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37273 } }
I20260808 08:29:00.157117 21379 leader_election.cc:288] T fca7001d55534ae8b1f488f402082e72 P 24236a034d2b493d8ebb6efaa9ca7dda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 11fd574efcfb4ccfb7e2ed56e3eed473 (127.22.235.194:40631), 40467ea65a9d4147a12e7706278dbaf2 (127.22.235.193:37223)
I20260808 08:29:00.160732 21343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:00.157318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.194:40631 (local address 127.22.235.195:58139)
0808 08:29:00.157522 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:00.157537 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:00.157637 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:00.157913 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:00.157916 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:00.157935 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:29:00.158062 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:00.158070 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:00.158817 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:00.158821 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:00.159613 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:00.159623 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:00.160240 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:00.160244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:00.160337 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:00.160409 (+ 72us) client_negotiation.cc:769] Sending connection context
0808 08:29:00.160473 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 08:29:00.160545 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":26}
I20260808 08:29:00.160732 21362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:00.157425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:58139 (local address 127.22.235.194:40631)
0808 08:29:00.157537 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:00.157542 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:00.157691 (+ 149us) server_negotiation.cc:407] Connection header received
0808 08:29:00.157760 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:00.157763 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:00.157821 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:00.157898 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:29:00.158168 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:00.158714 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:00.159735 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:00.160128 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:00.160221 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:00.160273 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:00.160508 (+ 235us) server_negotiation.cc:299] Negotiation successful
0808 08:29:00.160578 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 08:29:00.160751 21364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:00.157518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:58413 (local address 127.22.235.193:37223)
0808 08:29:00.157654 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:00.157659 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:00.157838 (+ 179us) server_negotiation.cc:407] Connection header received
0808 08:29:00.157954 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:00.157956 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:00.158013 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:00.158092 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:29:00.158337 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:00.158821 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:00.159781 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:00.160155 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:00.160264 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:00.160315 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:00.160557 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 08:29:00.160654 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 08:29:00.160754 21383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:00.157379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:37223 (local address 127.22.235.195:58413)
0808 08:29:00.157750 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:00.157763 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:00.157862 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:00.158109 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:00.158111 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:00.158129 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:29:00.158239 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:00.158245 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:00.158923 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:00.158925 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:00.159660 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:00.159668 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:00.160269 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:00.160273 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:00.160405 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:00.160422 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 08:29:00.160520 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 08:29:00.160580 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":23,"thread_start_us":139,"threads_started":1}
I20260808 08:29:00.161384 21043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fca7001d55534ae8b1f488f402082e72" candidate_uuid: "24236a034d2b493d8ebb6efaa9ca7dda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11fd574efcfb4ccfb7e2ed56e3eed473" is_pre_election: true
I20260808 08:29:00.161432 20839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fca7001d55534ae8b1f488f402082e72" candidate_uuid: "24236a034d2b493d8ebb6efaa9ca7dda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40467ea65a9d4147a12e7706278dbaf2" is_pre_election: true
I20260808 08:29:00.161700 21043 raft_consensus.cc:2454] T fca7001d55534ae8b1f488f402082e72 P 11fd574efcfb4ccfb7e2ed56e3eed473 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24236a034d2b493d8ebb6efaa9ca7dda in term 0.
I20260808 08:29:00.161711 20839 raft_consensus.cc:2454] T fca7001d55534ae8b1f488f402082e72 P 40467ea65a9d4147a12e7706278dbaf2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24236a034d2b493d8ebb6efaa9ca7dda in term 0.
I20260808 08:29:00.162147 21140 leader_election.cc:302] T fca7001d55534ae8b1f488f402082e72 P 24236a034d2b493d8ebb6efaa9ca7dda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24236a034d2b493d8ebb6efaa9ca7dda, 40467ea65a9d4147a12e7706278dbaf2; no voters:
I20260808 08:29:00.162387 21379 raft_consensus.cc:2781] T fca7001d55534ae8b1f488f402082e72 P 24236a034d2b493d8ebb6efaa9ca7dda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 08:29:00.162475 21379 raft_consensus.cc:482] T fca7001d55534ae8b1f488f402082e72 P 24236a034d2b493d8ebb6efaa9ca7dda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 08:29:00.162526 21379 raft_consensus.cc:3037] T fca7001d55534ae8b1f488f402082e72 P 24236a034d2b493d8ebb6efaa9ca7dda [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:29:00.163514 21379 raft_consensus.cc:504] T fca7001d55534ae8b1f488f402082e72 P 24236a034d2b493d8ebb6efaa9ca7dda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11fd574efcfb4ccfb7e2ed56e3eed473" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 40631 } } peers { permanent_uuid: "40467ea65a9d4147a12e7706278dbaf2" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 37223 } } peers { permanent_uuid: "24236a034d2b493d8ebb6efaa9ca7dda" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37273 } }
I20260808 08:29:00.163797 21379 leader_election.cc:288] T fca7001d55534ae8b1f488f402082e72 P 24236a034d2b493d8ebb6efaa9ca7dda [CANDIDATE]: Term 1 election: Requested vote from peers 11fd574efcfb4ccfb7e2ed56e3eed473 (127.22.235.194:40631), 40467ea65a9d4147a12e7706278dbaf2 (127.22.235.193:37223)
I20260808 08:29:00.164093 21043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fca7001d55534ae8b1f488f402082e72" candidate_uuid: "24236a034d2b493d8ebb6efaa9ca7dda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11fd574efcfb4ccfb7e2ed56e3eed473"
I20260808 08:29:00.164111 20839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fca7001d55534ae8b1f488f402082e72" candidate_uuid: "24236a034d2b493d8ebb6efaa9ca7dda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40467ea65a9d4147a12e7706278dbaf2"
I20260808 08:29:00.164211 20839 raft_consensus.cc:3037] T fca7001d55534ae8b1f488f402082e72 P 40467ea65a9d4147a12e7706278dbaf2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:29:00.164218 21043 raft_consensus.cc:3037] T fca7001d55534ae8b1f488f402082e72 P 11fd574efcfb4ccfb7e2ed56e3eed473 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:29:00.165217 20839 raft_consensus.cc:2454] T fca7001d55534ae8b1f488f402082e72 P 40467ea65a9d4147a12e7706278dbaf2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24236a034d2b493d8ebb6efaa9ca7dda in term 1.
I20260808 08:29:00.165230 21043 raft_consensus.cc:2454] T fca7001d55534ae8b1f488f402082e72 P 11fd574efcfb4ccfb7e2ed56e3eed473 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24236a034d2b493d8ebb6efaa9ca7dda in term 1.
I20260808 08:29:00.165509 21144 leader_election.cc:302] T fca7001d55534ae8b1f488f402082e72 P 24236a034d2b493d8ebb6efaa9ca7dda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11fd574efcfb4ccfb7e2ed56e3eed473, 24236a034d2b493d8ebb6efaa9ca7dda; no voters:
I20260808 08:29:00.165678 21379 raft_consensus.cc:2781] T fca7001d55534ae8b1f488f402082e72 P 24236a034d2b493d8ebb6efaa9ca7dda [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:29:00.165894 21379 raft_consensus.cc:686] T fca7001d55534ae8b1f488f402082e72 P 24236a034d2b493d8ebb6efaa9ca7dda [term 1 LEADER]: Becoming Leader. State: Replica: 24236a034d2b493d8ebb6efaa9ca7dda, State: Running, Role: LEADER
I20260808 08:29:00.166220 21379 consensus_queue.cc:237] T fca7001d55534ae8b1f488f402082e72 P 24236a034d2b493d8ebb6efaa9ca7dda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11fd574efcfb4ccfb7e2ed56e3eed473" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 40631 } } peers { permanent_uuid: "40467ea65a9d4147a12e7706278dbaf2" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 37223 } } peers { permanent_uuid: "24236a034d2b493d8ebb6efaa9ca7dda" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37273 } }
I20260808 08:29:00.169256 20606 catalog_manager.cc:5495] T fca7001d55534ae8b1f488f402082e72 P 24236a034d2b493d8ebb6efaa9ca7dda reported cstate change: term changed from 0 to 1, leader changed from <none> to 24236a034d2b493d8ebb6efaa9ca7dda (127.22.235.195). New cstate: current_term: 1 leader_uuid: "24236a034d2b493d8ebb6efaa9ca7dda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11fd574efcfb4ccfb7e2ed56e3eed473" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 40631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40467ea65a9d4147a12e7706278dbaf2" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 37223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24236a034d2b493d8ebb6efaa9ca7dda" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 37273 } health_report { overall_health: HEALTHY } } }
I20260808 08:29:00.209017 21365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:00.205660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53782 (local address 127.22.235.195:37273)
0808 08:29:00.205777 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:00.205781 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:00.205793 (+ 12us) server_negotiation.cc:407] Connection header received
0808 08:29:00.205869 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:00.205871 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:00.205925 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:00.206012 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:29:00.206198 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:00.206774 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:00.207648 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:00.207852 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:00.207933 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:00.208180 (+ 247us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:29:00.208688 (+ 508us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 08:29:00.208780 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:00.208847 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 08:29:00.208896 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 08:29:00.214872 21043 raft_consensus.cc:1261] T fca7001d55534ae8b1f488f402082e72 P 11fd574efcfb4ccfb7e2ed56e3eed473 [term 1 FOLLOWER]: Refusing update from remote peer 24236a034d2b493d8ebb6efaa9ca7dda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:29:00.214876 20839 raft_consensus.cc:1261] T fca7001d55534ae8b1f488f402082e72 P 40467ea65a9d4147a12e7706278dbaf2 [term 1 FOLLOWER]: Refusing update from remote peer 24236a034d2b493d8ebb6efaa9ca7dda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:29:00.215559 21384 consensus_queue.cc:1035] T fca7001d55534ae8b1f488f402082e72 P 24236a034d2b493d8ebb6efaa9ca7dda [LEADER]: Connected to new peer: Peer: permanent_uuid: "11fd574efcfb4ccfb7e2ed56e3eed473" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 40631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:29:00.215754 21379 consensus_queue.cc:1035] T fca7001d55534ae8b1f488f402082e72 P 24236a034d2b493d8ebb6efaa9ca7dda [LEADER]: Connected to new peer: Peer: permanent_uuid: "40467ea65a9d4147a12e7706278dbaf2" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 37223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:29:00.224538 21394 mvcc.cc:205] Tried to move back new op lower bound from 7316184023912357888 to 7316184023723581440. Current Snapshot: MvccSnapshot[applied={T|T < 7316184023912357888}]
I20260808 08:29:00.230856 20606 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40932:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:29:00.231027 20606 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40932:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:29:00.233647 20606 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 53514b1c95124e4185b238debf05523a: Sending DeleteTablet for 3 replicas of tablet fca7001d55534ae8b1f488f402082e72
I20260808 08:29:00.234398 21016 tablet_service.cc:1514] Processing DeleteTablet for tablet fca7001d55534ae8b1f488f402082e72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:29:00 UTC) from {username='slave'} at 127.0.0.1:36434
I20260808 08:29:00.234412 20803 tablet_service.cc:1514] Processing DeleteTablet for tablet fca7001d55534ae8b1f488f402082e72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:29:00 UTC) from {username='slave'} at 127.0.0.1:54526
I20260808 08:29:00.234913 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 20696
I20260808 08:29:00.234998 21230 tablet_service.cc:1514] Processing DeleteTablet for tablet fca7001d55534ae8b1f488f402082e72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:29:00 UTC) from {username='slave'} at 127.0.0.1:53780
I20260808 08:29:00.235579 21411 tablet_replica.cc:330] T fca7001d55534ae8b1f488f402082e72 P 11fd574efcfb4ccfb7e2ed56e3eed473: stopping tablet replica
I20260808 08:29:00.235948 21411 raft_consensus.cc:2229] T fca7001d55534ae8b1f488f402082e72 P 11fd574efcfb4ccfb7e2ed56e3eed473 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 08:29:00.236302 21411 raft_consensus.cc:2258] T fca7001d55534ae8b1f488f402082e72 P 11fd574efcfb4ccfb7e2ed56e3eed473 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:29:00.237958 21411 ts_tablet_manager.cc:1905] T fca7001d55534ae8b1f488f402082e72 P 11fd574efcfb4ccfb7e2ed56e3eed473: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:29:00.241168 21411 ts_tablet_manager.cc:1918] T fca7001d55534ae8b1f488f402082e72 P 11fd574efcfb4ccfb7e2ed56e3eed473: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 08:29:00.241281 21411 log.cc:1196] T fca7001d55534ae8b1f488f402082e72 P 11fd574efcfb4ccfb7e2ed56e3eed473: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-1/wal/wals/fca7001d55534ae8b1f488f402082e72
I20260808 08:29:00.241572 21411 ts_tablet_manager.cc:1939] T fca7001d55534ae8b1f488f402082e72 P 11fd574efcfb4ccfb7e2ed56e3eed473: Deleting consensus metadata
I20260808 08:29:00.242627 20584 catalog_manager.cc:4841] TS 11fd574efcfb4ccfb7e2ed56e3eed473 (127.22.235.194:40631): tablet fca7001d55534ae8b1f488f402082e72 (table test-table [id=e570c86c4dd14d5db1d5a88268968cc7]) successfully deleted
I20260808 08:29:00.249553 21414 tablet_replica.cc:330] T fca7001d55534ae8b1f488f402082e72 P 24236a034d2b493d8ebb6efaa9ca7dda: stopping tablet replica
I20260808 08:29:00.249958 21414 raft_consensus.cc:2229] T fca7001d55534ae8b1f488f402082e72 P 24236a034d2b493d8ebb6efaa9ca7dda [term 1 LEADER]: Raft consensus shutting down.
I20260808 08:29:00.250526 21414 raft_consensus.cc:2258] T fca7001d55534ae8b1f488f402082e72 P 24236a034d2b493d8ebb6efaa9ca7dda [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:29:00.252424 21414 ts_tablet_manager.cc:1905] T fca7001d55534ae8b1f488f402082e72 P 24236a034d2b493d8ebb6efaa9ca7dda: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 08:29:00.253639 20582 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.235.193:37223 (error 108)
I20260808 08:29:00.253849 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 20908
I20260808 08:29:00.282651 21361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:00.282290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:37223 (local address 127.0.0.1:54538)
0808 08:29:00.282483 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:00.282555 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.193:37223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
W20260808 08:29:00.282833 20582 catalog_manager.cc:4584] TS 40467ea65a9d4147a12e7706278dbaf2 (127.22.235.193:37223): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fca7001d55534ae8b1f488f402082e72: Network error: Client connection negotiation failed: client connection to 127.22.235.193:37223: connect: Connection refused (error 111)
I20260808 08:29:00.285758 21414 ts_tablet_manager.cc:1918] T fca7001d55534ae8b1f488f402082e72 P 24236a034d2b493d8ebb6efaa9ca7dda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 08:29:00.285876 21414 log.cc:1196] T fca7001d55534ae8b1f488f402082e72 P 24236a034d2b493d8ebb6efaa9ca7dda: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786177493204855-23471-0/minicluster-data/ts-2/wal/wals/fca7001d55534ae8b1f488f402082e72
I20260808 08:29:00.286151 21414 ts_tablet_manager.cc:1939] T fca7001d55534ae8b1f488f402082e72 P 24236a034d2b493d8ebb6efaa9ca7dda: Deleting consensus metadata
I20260808 08:29:00.287436 20584 catalog_manager.cc:4841] TS 24236a034d2b493d8ebb6efaa9ca7dda (127.22.235.195:37273): tablet fca7001d55534ae8b1f488f402082e72 (table test-table [id=e570c86c4dd14d5db1d5a88268968cc7]) successfully deleted
I20260808 08:29:00.293923 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 21120
I20260808 08:29:00.307850 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 20561
2026-08-08T08:29:00Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1180 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 08:29:00 dist-test-slave-f7th krb5kdc[21422](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 08:29:00 dist-test-slave-f7th krb5kdc[21422](info): set up 2 sockets
Aug 08 08:29:00 dist-test-slave-f7th krb5kdc[21422](info): commencing operation
krb5kdc: starting...
W20260808 08:29:01.467272 23471 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Aug 08 08:29:01 dist-test-slave-f7th krb5kdc[21422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177741, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T08:29:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:29:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.235.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
I20260808 08:29:01.586933 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:39449
--webserver_interface=127.22.235.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:33395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:39449
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:29:01.698783 21441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:29:01.699035 21441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:29:01.699074 21441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:29:01.703016 21441 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:29:01.703069 21441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:29:01.703087 21441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:29:01.703102 21441 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:29:01.703116 21441 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:29:01.708246 21441 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:33395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:39449
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:39449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.235.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:29:01.709234 21441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:29:01.710176 21441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:29:01.718375 21451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:01.718437 21441 server_base.cc:1034] running on GCE node
W20260808 08:29:01.718384 21456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:01.718384 21446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:01.719218 21441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:29:01.720238 21441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:29:01.721385 21441 hybrid_clock.cc:648] HybridClock initialized: now 1786177741721366 us; error 29 us; skew 500 ppm
Aug 08 08:29:01 dist-test-slave-f7th krb5kdc[21422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177741, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:29:01.727514 21441 init.cc:377] Logged in from keytab as kudu/127.22.235.254@KRBTEST.COM (short username kudu)
I20260808 08:29:01.728415 21441 webserver.cc:460] Webserver started at http://127.22.235.254:33361/ using document root <none> and password file <none>
I20260808 08:29:01.728981 21441 fs_manager.cc:359] Metadata directory not provided
I20260808 08:29:01.729058 21441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:29:01.729264 21441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:29:01.731329 21441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "cdbafa9845a4418db97b983bc6f3e4c7"
format_stamp: "Formatted at 2026-08-08 08:29:01 on dist-test-slave-f7th"
server_key: "072907aa63061fb545afab137ca1394a"
server_key_iv: "9b4590319fd105bbc85b04eab21cea4a"
server_key_version: "encryptionkey@0"
I20260808 08:29:01.731858 21441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "cdbafa9845a4418db97b983bc6f3e4c7"
format_stamp: "Formatted at 2026-08-08 08:29:01 on dist-test-slave-f7th"
server_key: "072907aa63061fb545afab137ca1394a"
server_key_iv: "9b4590319fd105bbc85b04eab21cea4a"
server_key_version: "encryptionkey@0"
I20260808 08:29:01.735605 21441 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260808 08:29:01.738101 21467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:01.739316 21441 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 08:29:01.739436 21441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "cdbafa9845a4418db97b983bc6f3e4c7"
format_stamp: "Formatted at 2026-08-08 08:29:01 on dist-test-slave-f7th"
server_key: "072907aa63061fb545afab137ca1394a"
server_key_iv: "9b4590319fd105bbc85b04eab21cea4a"
server_key_version: "encryptionkey@0"
I20260808 08:29:01.739537 21441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:29:01.769284 21441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:29:01.772055 21441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:29:01.772228 21441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:29:01.778996 21441 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.254:39449
I20260808 08:29:01.779044 21543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.254:39449 every 8 connection(s)
I20260808 08:29:01.779956 21441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:29:01.782712 21545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:29:01.783018 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 21441
I20260808 08:29:01.783165 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:29:01.783481 23471 external_mini_cluster.cc:1428] Setting key 2d032d80492c359f6f858139568b1360
I20260808 08:29:01.789495 21545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdbafa9845a4418db97b983bc6f3e4c7: Bootstrap starting.
I20260808 08:29:01.791720 21545 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cdbafa9845a4418db97b983bc6f3e4c7: Neither blocks nor log segments found. Creating new log.
Aug 08 08:29:01 dist-test-slave-f7th krb5kdc[21422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177741, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:29:01.792482 21545 log.cc:824] T 00000000000000000000000000000000 P cdbafa9845a4418db97b983bc6f3e4c7: Log is configured to *not* fsync() on all Append() calls
I20260808 08:29:01.794579 21545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdbafa9845a4418db97b983bc6f3e4c7: No bootstrap required, opened a new log
I20260808 08:29:01.797425 21548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:01.784920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49150 (local address 127.22.235.254:39449)
0808 08:29:01.785361 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:01.785376 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:01.785410 (+ 34us) server_negotiation.cc:407] Connection header received
0808 08:29:01.786095 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:01.786114 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:01.786454 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:01.786763 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:01.787572 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:01.788549 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:01.789544 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:01.790104 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:01.790262 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:01.792695 (+ 2433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:01.792700 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:01.792714 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:01.792743 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:01.794363 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:01.794791 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:01.794798 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:01.794804 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:01.794958 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:01.795199 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:01.795202 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:01.795204 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:01.795710 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:01.795905 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:01.796096 (+ 191us) server_negotiation.cc:299] Negotiation successful
0808 08:29:01.796404 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":125,"threads_started":1}
I20260808 08:29:01.798807 21545 raft_consensus.cc:348] T 00000000000000000000000000000000 P cdbafa9845a4418db97b983bc6f3e4c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdbafa9845a4418db97b983bc6f3e4c7" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 39449 } }
I20260808 08:29:01.799019 21545 raft_consensus.cc:374] T 00000000000000000000000000000000 P cdbafa9845a4418db97b983bc6f3e4c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:29:01.799063 21545 raft_consensus.cc:729] T 00000000000000000000000000000000 P cdbafa9845a4418db97b983bc6f3e4c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdbafa9845a4418db97b983bc6f3e4c7, State: Initialized, Role: FOLLOWER
I20260808 08:29:01.799467 21545 consensus_queue.cc:260] T 00000000000000000000000000000000 P cdbafa9845a4418db97b983bc6f3e4c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdbafa9845a4418db97b983bc6f3e4c7" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 39449 } }
I20260808 08:29:01.799614 21545 raft_consensus.cc:388] T 00000000000000000000000000000000 P cdbafa9845a4418db97b983bc6f3e4c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:29:01.799681 21545 raft_consensus.cc:482] T 00000000000000000000000000000000 P cdbafa9845a4418db97b983bc6f3e4c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:29:01.799772 21545 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cdbafa9845a4418db97b983bc6f3e4c7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:29:01.800649 21545 raft_consensus.cc:504] T 00000000000000000000000000000000 P cdbafa9845a4418db97b983bc6f3e4c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdbafa9845a4418db97b983bc6f3e4c7" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 39449 } }
I20260808 08:29:01.801005 21545 leader_election.cc:302] T 00000000000000000000000000000000 P cdbafa9845a4418db97b983bc6f3e4c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cdbafa9845a4418db97b983bc6f3e4c7; no voters:
I20260808 08:29:01.801275 21545 leader_election.cc:288] T 00000000000000000000000000000000 P cdbafa9845a4418db97b983bc6f3e4c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:29:01.801375 21556 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cdbafa9845a4418db97b983bc6f3e4c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:29:01.801586 21556 raft_consensus.cc:686] T 00000000000000000000000000000000 P cdbafa9845a4418db97b983bc6f3e4c7 [term 1 LEADER]: Becoming Leader. State: Replica: cdbafa9845a4418db97b983bc6f3e4c7, State: Running, Role: LEADER
I20260808 08:29:01.801877 21556 consensus_queue.cc:237] T 00000000000000000000000000000000 P cdbafa9845a4418db97b983bc6f3e4c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdbafa9845a4418db97b983bc6f3e4c7" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 39449 } }
I20260808 08:29:01.802551 21545 sys_catalog.cc:564] T 00000000000000000000000000000000 P cdbafa9845a4418db97b983bc6f3e4c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:29:01.803591 21557 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdbafa9845a4418db97b983bc6f3e4c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cdbafa9845a4418db97b983bc6f3e4c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdbafa9845a4418db97b983bc6f3e4c7" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 39449 } } }
I20260808 08:29:01.803720 21557 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdbafa9845a4418db97b983bc6f3e4c7 [sys.catalog]: This master's current role is: LEADER
I20260808 08:29:01.803989 21563 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdbafa9845a4418db97b983bc6f3e4c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cdbafa9845a4418db97b983bc6f3e4c7. Latest consensus state: current_term: 1 leader_uuid: "cdbafa9845a4418db97b983bc6f3e4c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdbafa9845a4418db97b983bc6f3e4c7" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 39449 } } }
I20260808 08:29:01.804083 21563 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdbafa9845a4418db97b983bc6f3e4c7 [sys.catalog]: This master's current role is: LEADER
I20260808 08:29:01.804193 21581 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:29:01.807529 21581 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:29:01.812814 21581 catalog_manager.cc:1347] Generated new cluster ID: e7570c59a5804865ba3b3b97200e49c9
I20260808 08:29:01.812881 21581 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:29:01.822956 21581 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:29:01.823963 21581 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:29:01.828114 21581 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cdbafa9845a4418db97b983bc6f3e4c7: Generated new TSK 0
I20260808 08:29:01.828787 21581 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.235.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab.
Entry for principal kudu/127.22.235.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab.
I20260808 08:29:01.861311 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.193:0
--local_ip_for_outbound_sockets=127.22.235.193
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:39449
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab
--principal=kudu/127.22.235.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:33395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:29:02.001127 21597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:29:02.001370 21597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:29:02.001411 21597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:29:02.005525 21597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:29:02.005582 21597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:29:02.005648 21597 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.193
I20260808 08:29:02.010831 21597 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:33395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab
--principal=kudu/127.22.235.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:39449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21597
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.193
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:29:02.011854 21597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:29:02.012935 21597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:29:02.022243 21605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:02.022260 21606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:02.022248 21608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:02.022636 21597 server_base.cc:1034] running on GCE node
I20260808 08:29:02.023017 21597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:29:02.023696 21597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:29:02.024865 21597 hybrid_clock.cc:648] HybridClock initialized: now 1786177742024836 us; error 39 us; skew 500 ppm
Aug 08 08:29:02 dist-test-slave-f7th krb5kdc[21422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177742, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:29:02.027537 21597 init.cc:377] Logged in from keytab as kudu/127.22.235.193@KRBTEST.COM (short username kudu)
I20260808 08:29:02.028379 21597 webserver.cc:460] Webserver started at http://127.22.235.193:35389/ using document root <none> and password file <none>
I20260808 08:29:02.028960 21597 fs_manager.cc:359] Metadata directory not provided
I20260808 08:29:02.029022 21597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:29:02.029217 21597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:29:02.031160 21597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-0/data/instance:
uuid: "3d1c33d59ab44ac6a9791e113978fec6"
format_stamp: "Formatted at 2026-08-08 08:29:02 on dist-test-slave-f7th"
server_key: "4a63cc72a5bdd5d57f7b9d6e9b88d750"
server_key_iv: "7dd26d5a73ca8cd46e82d29e5d337a3e"
server_key_version: "encryptionkey@0"
I20260808 08:29:02.031656 21597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance:
uuid: "3d1c33d59ab44ac6a9791e113978fec6"
format_stamp: "Formatted at 2026-08-08 08:29:02 on dist-test-slave-f7th"
server_key: "4a63cc72a5bdd5d57f7b9d6e9b88d750"
server_key_iv: "7dd26d5a73ca8cd46e82d29e5d337a3e"
server_key_version: "encryptionkey@0"
I20260808 08:29:02.035254 21597 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 08:29:02.037576 21617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:02.038782 21597 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:29:02.038877 21597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-0/wal
uuid: "3d1c33d59ab44ac6a9791e113978fec6"
format_stamp: "Formatted at 2026-08-08 08:29:02 on dist-test-slave-f7th"
server_key: "4a63cc72a5bdd5d57f7b9d6e9b88d750"
server_key_iv: "7dd26d5a73ca8cd46e82d29e5d337a3e"
server_key_version: "encryptionkey@0"
I20260808 08:29:02.038972 21597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:29:02.058051 21597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:29:02.060810 21597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:29:02.060990 21597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:29:02.061596 21597 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:29:02.062434 21597 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:29:02.062479 21597 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:02.062528 21597 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:29:02.062558 21597 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:02.070657 21597 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.193:45581
I20260808 08:29:02.070690 21789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.193:45581 every 8 connection(s)
I20260808 08:29:02.071523 21597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
I20260808 08:29:02.078660 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 21597
I20260808 08:29:02.078780 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance
I20260808 08:29:02.079092 23471 external_mini_cluster.cc:1428] Setting key 6049e6588f97ffff5551b744b1a2fd7a
Aug 08 08:29:02 dist-test-slave-f7th krb5kdc[21422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177742, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:29:02.085283 21548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:02.073303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:41663 (local address 127.22.235.254:39449)
0808 08:29:02.073443 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:02.073447 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:02.074252 (+ 805us) server_negotiation.cc:407] Connection header received
0808 08:29:02.075545 (+ 1293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:02.075550 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:02.075608 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:02.075682 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:02.076852 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:02.077373 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:02.078505 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:02.078779 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:02.078884 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:02.081810 (+ 2926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:02.081813 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:02.081815 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:02.081840 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:02.083262 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:02.083741 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:02.083745 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:02.083747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:02.083843 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:02.084191 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:02.084195 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:02.084197 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:02.084376 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:02.084453 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:02.085052 (+ 599us) server_negotiation.cc:299] Negotiation successful
0808 08:29:02.085163 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 08:29:02.086431 21792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:02.073654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:39449 (local address 127.22.235.193:41663)
0808 08:29:02.074109 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:02.074151 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:02.075330 (+ 1179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:02.075869 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:02.075877 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:02.076222 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:29:02.076706 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:02.076720 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:02.077497 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:02.077500 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:02.078376 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:02.078387 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:02.079035 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:02.079037 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:02.079201 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:02.079881 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:29:02.079901 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:29:02.081649 (+ 1748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:29:02.083375 (+ 1726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:29:02.083380 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:29:02.083392 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:29:02.083651 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:29:02.083939 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:29:02.083941 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:29:02.083943 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:29:02.084100 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:29:02.084480 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:29:02.084486 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:29:02.084815 (+ 329us) client_negotiation.cc:769] Sending connection context
0808 08:29:02.085026 (+ 211us) client_negotiation.cc:240] Negotiation successful
0808 08:29:02.085354 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":104,"threads_started":1}
I20260808 08:29:02.087677 21790 heartbeater.cc:347] Connected to a master server at 127.22.235.254:39449
I20260808 08:29:02.087911 21790 heartbeater.cc:464] Registering TS with master...
I20260808 08:29:02.088372 21790 heartbeater.cc:511] Master 127.22.235.254:39449 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.22.235.194@KRBTEST.COM; defaulting to no policy
I20260808 08:29:02.090073 21494 ts_manager.cc:194] Registered new tserver with Master: 3d1c33d59ab44ac6a9791e113978fec6 (127.22.235.193:45581)
I20260808 08:29:02.091265 21494 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.193@KRBTEST.COM'} at 127.22.235.193:41663
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab.
Entry for principal kudu/127.22.235.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab.
I20260808 08:29:02.107311 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.194:0
--local_ip_for_outbound_sockets=127.22.235.194
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:39449
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab
--principal=kudu/127.22.235.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:33395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:29:02.217490 21797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:29:02.217741 21797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:29:02.217783 21797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:29:02.221956 21797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:29:02.222019 21797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:29:02.222090 21797 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.194
I20260808 08:29:02.227303 21797 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:33395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab
--principal=kudu/127.22.235.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:39449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21797
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.194
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:29:02.228353 21797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:29:02.229435 21797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:29:02.238961 21807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:02.238900 21803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:02.238903 21804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:02.239181 21797 server_base.cc:1034] running on GCE node
I20260808 08:29:02.239555 21797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:29:02.240171 21797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:29:02.241324 21797 hybrid_clock.cc:648] HybridClock initialized: now 1786177742241300 us; error 35 us; skew 500 ppm
Aug 08 08:29:02 dist-test-slave-f7th krb5kdc[21422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177742, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:29:02.244071 21797 init.cc:377] Logged in from keytab as kudu/127.22.235.194@KRBTEST.COM (short username kudu)
I20260808 08:29:02.244943 21797 webserver.cc:460] Webserver started at http://127.22.235.194:37289/ using document root <none> and password file <none>
I20260808 08:29:02.245469 21797 fs_manager.cc:359] Metadata directory not provided
I20260808 08:29:02.245527 21797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:29:02.245698 21797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:29:02.247730 21797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-1/data/instance:
uuid: "e353b4afab41427690334c8eadeb8d17"
format_stamp: "Formatted at 2026-08-08 08:29:02 on dist-test-slave-f7th"
server_key: "efd3ffa2e1e81bceae0961dc5539a7d6"
server_key_iv: "e0b2bf109eedcf8f1f72cc4bea9571a6"
server_key_version: "encryptionkey@0"
I20260808 08:29:02.248239 21797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance:
uuid: "e353b4afab41427690334c8eadeb8d17"
format_stamp: "Formatted at 2026-08-08 08:29:02 on dist-test-slave-f7th"
server_key: "efd3ffa2e1e81bceae0961dc5539a7d6"
server_key_iv: "e0b2bf109eedcf8f1f72cc4bea9571a6"
server_key_version: "encryptionkey@0"
I20260808 08:29:02.251861 21797 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 08:29:02.254290 21821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:02.255501 21797 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260808 08:29:02.255611 21797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-1/wal
uuid: "e353b4afab41427690334c8eadeb8d17"
format_stamp: "Formatted at 2026-08-08 08:29:02 on dist-test-slave-f7th"
server_key: "efd3ffa2e1e81bceae0961dc5539a7d6"
server_key_iv: "e0b2bf109eedcf8f1f72cc4bea9571a6"
server_key_version: "encryptionkey@0"
I20260808 08:29:02.255705 21797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:29:02.269407 21797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:29:02.272447 21797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:29:02.272686 21797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:29:02.273262 21797 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:29:02.274081 21797 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:29:02.274129 21797 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:02.274183 21797 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:29:02.274209 21797 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:02.282789 21797 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.194:40049
I20260808 08:29:02.282814 21994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.194:40049 every 8 connection(s)
I20260808 08:29:02.283643 21797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
Aug 08 08:29:02 dist-test-slave-f7th krb5kdc[21422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177742, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.194@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:29:02.293582 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 21797
I20260808 08:29:02.293707 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance
I20260808 08:29:02.293969 23471 external_mini_cluster.cc:1428] Setting key c5f9d588cbc231e484234bf67f138dfc
I20260808 08:29:02.296372 21548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:02.285383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:50545 (local address 127.22.235.254:39449)
0808 08:29:02.285525 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:02.285530 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:02.286352 (+ 822us) server_negotiation.cc:407] Connection header received
0808 08:29:02.287584 (+ 1232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:02.287587 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:02.287633 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:02.287704 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:02.288824 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:02.289348 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:02.290385 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:02.290595 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:02.290666 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:02.292933 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:02.292936 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:02.292938 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:02.292977 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:02.294350 (+ 1373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:02.294895 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:02.294899 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:02.294900 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:02.294988 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:02.295320 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:02.295324 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:02.295325 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:02.295525 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:02.295618 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:02.296096 (+ 478us) server_negotiation.cc:299] Negotiation successful
0808 08:29:02.296215 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 08:29:02.297192 21998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:02.285775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:39449 (local address 127.22.235.194:50545)
0808 08:29:02.286206 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:02.286246 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:02.287395 (+ 1149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:02.287848 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:02.287854 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:02.288202 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:29:02.288705 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:02.288721 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:02.289440 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:02.289443 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:02.290299 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:02.290308 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:02.290700 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:02.290702 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:02.290841 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:02.291326 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:29:02.291341 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:29:02.292773 (+ 1432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:29:02.294459 (+ 1686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:29:02.294479 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:29:02.294495 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:29:02.294801 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:29:02.295070 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:29:02.295072 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:29:02.295073 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:29:02.295229 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:29:02.295631 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:29:02.295638 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:29:02.295863 (+ 225us) client_negotiation.cc:769] Sending connection context
0808 08:29:02.296025 (+ 162us) client_negotiation.cc:240] Negotiation successful
0808 08:29:02.296350 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":108,"threads_started":1}
I20260808 08:29:02.298537 21995 heartbeater.cc:347] Connected to a master server at 127.22.235.254:39449
I20260808 08:29:02.298761 21995 heartbeater.cc:464] Registering TS with master...
I20260808 08:29:02.299220 21995 heartbeater.cc:511] Master 127.22.235.254:39449 requested a full tablet report, sending...
I20260808 08:29:02.300276 21494 ts_manager.cc:194] Registered new tserver with Master: e353b4afab41427690334c8eadeb8d17 (127.22.235.194:40049)
I20260808 08:29:02.301060 21494 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.194@KRBTEST.COM'} at 127.22.235.194:50545
WARNING: no policy specified for kudu/127.22.235.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab.
Entry for principal kudu/127.22.235.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab.
I20260808 08:29:02.320742 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.195:0
--local_ip_for_outbound_sockets=127.22.235.195
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:39449
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab
--principal=kudu/127.22.235.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:33395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:29:02.431743 22001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:29:02.431982 22001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:29:02.432037 22001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:29:02.435968 22001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:29:02.436023 22001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:29:02.436085 22001 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.195
I20260808 08:29:02.441176 22001 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:33395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab
--principal=kudu/127.22.235.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:39449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.195
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:29:02.442135 22001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:29:02.443181 22001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:29:02.452553 22010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:02.452674 22001 server_base.cc:1034] running on GCE node
W20260808 08:29:02.452582 22008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:02.452551 22007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:02.453153 22001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:29:02.453751 22001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:29:02.454903 22001 hybrid_clock.cc:648] HybridClock initialized: now 1786177742454881 us; error 35 us; skew 500 ppm
Aug 08 08:29:02 dist-test-slave-f7th krb5kdc[21422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177742, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:29:02.457469 22001 init.cc:377] Logged in from keytab as kudu/127.22.235.195@KRBTEST.COM (short username kudu)
I20260808 08:29:02.458491 22001 webserver.cc:460] Webserver started at http://127.22.235.195:34215/ using document root <none> and password file <none>
I20260808 08:29:02.459034 22001 fs_manager.cc:359] Metadata directory not provided
I20260808 08:29:02.459095 22001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:29:02.459297 22001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:29:02.461418 22001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-2/data/instance:
uuid: "a916a34e157c446a9282d8a4533dba81"
format_stamp: "Formatted at 2026-08-08 08:29:02 on dist-test-slave-f7th"
server_key: "da7e5b2286140dc8da43e19b2aa91e3c"
server_key_iv: "34fd56fc569ff422a6ec3167994992ed"
server_key_version: "encryptionkey@0"
I20260808 08:29:02.461956 22001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance:
uuid: "a916a34e157c446a9282d8a4533dba81"
format_stamp: "Formatted at 2026-08-08 08:29:02 on dist-test-slave-f7th"
server_key: "da7e5b2286140dc8da43e19b2aa91e3c"
server_key_iv: "34fd56fc569ff422a6ec3167994992ed"
server_key_version: "encryptionkey@0"
I20260808 08:29:02.465551 22001 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 08:29:02.467835 22021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:02.469043 22001 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 08:29:02.469144 22001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-2/wal
uuid: "a916a34e157c446a9282d8a4533dba81"
format_stamp: "Formatted at 2026-08-08 08:29:02 on dist-test-slave-f7th"
server_key: "da7e5b2286140dc8da43e19b2aa91e3c"
server_key_iv: "34fd56fc569ff422a6ec3167994992ed"
server_key_version: "encryptionkey@0"
I20260808 08:29:02.469240 22001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:29:02.498625 22001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:29:02.501515 22001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:29:02.501695 22001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:29:02.502305 22001 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:29:02.503248 22001 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:29:02.503298 22001 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:02.503342 22001 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:29:02.503371 22001 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:02.512138 22001 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.195:34835
I20260808 08:29:02.512171 22200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.195:34835 every 8 connection(s)
I20260808 08:29:02.513046 22001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
I20260808 08:29:02.517010 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 22001
I20260808 08:29:02.517134 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance
I20260808 08:29:02.517397 23471 external_mini_cluster.cc:1428] Setting key f0547108ac3e27e2f069cbb100833416
Aug 08 08:29:02 dist-test-slave-f7th krb5kdc[21422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177742, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.195@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:29:02.525918 21548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:02.514782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:54429 (local address 127.22.235.254:39449)
0808 08:29:02.514947 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:02.514952 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:02.515826 (+ 874us) server_negotiation.cc:407] Connection header received
0808 08:29:02.517171 (+ 1345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:02.517175 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:02.517233 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:02.517310 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:02.518674 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:02.519207 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:02.520382 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:02.520602 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:02.520720 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:02.523031 (+ 2311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:02.523034 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:02.523036 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:02.523062 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:02.524163 (+ 1101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:02.524607 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:02.524610 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:02.524611 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:02.524722 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:02.525053 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:02.525055 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:02.525056 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:02.525218 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:02.525290 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:02.525701 (+ 411us) server_negotiation.cc:299] Negotiation successful
0808 08:29:02.525798 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 08:29:02.526722 22205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:02.515158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:39449 (local address 127.22.235.195:54429)
0808 08:29:02.515647 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:02.515704 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:02.516953 (+ 1249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:02.517518 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:02.517528 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:02.517977 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:29:02.518540 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:02.518562 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:02.519329 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:02.519333 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:02.520259 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:02.520270 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:02.520761 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:02.520765 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:02.520942 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:02.521431 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:29:02.521447 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:29:02.522848 (+ 1401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:29:02.524265 (+ 1417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:29:02.524270 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:29:02.524281 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:29:02.524537 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:29:02.524810 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:29:02.524813 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:29:02.524815 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:29:02.524969 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:29:02.525288 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:29:02.525293 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:29:02.525519 (+ 226us) client_negotiation.cc:769] Sending connection context
0808 08:29:02.525670 (+ 151us) client_negotiation.cc:240] Negotiation successful
0808 08:29:02.525907 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":113,"threads_started":1}
I20260808 08:29:02.527777 22202 heartbeater.cc:347] Connected to a master server at 127.22.235.254:39449
I20260808 08:29:02.528010 22202 heartbeater.cc:464] Registering TS with master...
I20260808 08:29:02.528477 22202 heartbeater.cc:511] Master 127.22.235.254:39449 requested a full tablet report, sending...
I20260808 08:29:02.529496 21494 ts_manager.cc:194] Registered new tserver with Master: a916a34e157c446a9282d8a4533dba81 (127.22.235.195:34835)
I20260808 08:29:02.530155 21494 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.195@KRBTEST.COM'} at 127.22.235.195:54429
I20260808 08:29:02.531037 23471 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 08:29:02.540530 21548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:02.533190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49160 (local address 127.22.235.254:39449)
0808 08:29:02.533308 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:02.533312 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:02.533482 (+ 170us) server_negotiation.cc:407] Connection header received
0808 08:29:02.533599 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:02.533602 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:02.533643 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:02.533708 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:02.534407 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:02.534914 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:02.535726 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:02.535955 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:02.536042 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:02.537632 (+ 1590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:02.537637 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:02.537639 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:02.537666 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:02.538763 (+ 1097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:02.539286 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:02.539289 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:02.539291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:02.539373 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:02.539680 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:02.539683 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:02.539685 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:02.539887 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:02.540025 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:02.540283 (+ 258us) server_negotiation.cc:299] Negotiation successful
0808 08:29:02.540394 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 08:29:02.543303 21494 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49160:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 08:29:02.546479 21494 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 08:29:02.561136 22222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:02.555518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55560 (local address 127.22.235.194:40049)
0808 08:29:02.555953 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:02.555965 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:02.555989 (+ 24us) server_negotiation.cc:407] Connection header received
0808 08:29:02.556064 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:02.556069 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:02.556269 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:02.556421 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:29:02.556808 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:02.557979 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:02.559485 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:02.560267 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:02.560655 (+ 388us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:02.560785 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:02.560879 (+ 94us) server_negotiation.cc:299] Negotiation successful
0808 08:29:02.560973 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":129,"threads_started":1}
I20260808 08:29:02.561182 22217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:02.554784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.194:40049 (local address 127.0.0.1:55560)
0808 08:29:02.555206 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:02.555221 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:02.555310 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:02.556437 (+ 1127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:02.556443 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:02.556467 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:29:02.556677 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:02.556685 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:02.558106 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:02.558109 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:02.559366 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:02.559376 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:02.560403 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:02.560407 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:02.560645 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:02.560725 (+ 80us) client_negotiation.cc:769] Sending connection context
0808 08:29:02.560851 (+ 126us) client_negotiation.cc:240] Negotiation successful
0808 08:29:02.561000 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"mutex_wait_us":173,"spinlock_wait_cycles":1280,"thread_start_us":119,"threads_started":1}
I20260808 08:29:02.561477 22219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:02.554858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33944 (local address 127.22.235.193:45581)
0808 08:29:02.556034 (+ 1176us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:02.556051 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:02.556074 (+ 23us) server_negotiation.cc:407] Connection header received
0808 08:29:02.556150 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:02.556155 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:02.556345 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:02.556664 (+ 319us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:29:02.556819 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:02.558015 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:02.559490 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:02.560262 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:02.560410 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:02.560562 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:02.561247 (+ 685us) server_negotiation.cc:299] Negotiation successful
0808 08:29:02.561354 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1073,"thread_start_us":84,"threads_started":1}
I20260808 08:29:02.561157 22218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:02.554715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:45581 (local address 127.0.0.1:33944)
0808 08:29:02.555137 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:02.555186 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:02.555309 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:02.556491 (+ 1182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:02.556494 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:02.556518 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:29:02.556712 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:02.556719 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:02.558181 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:02.558185 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:02.559376 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:02.559387 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:02.560403 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:02.560407 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:02.560583 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:02.560720 (+ 137us) client_negotiation.cc:769] Sending connection context
0808 08:29:02.560834 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 08:29:02.561001 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":247,"spinlock_wait_cycles":2688,"thread_start_us":74,"threads_started":1}
I20260808 08:29:02.561587 22221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:02.555395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.195:34835 (local address 127.0.0.1:39660)
0808 08:29:02.555618 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:02.555631 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:02.555718 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:02.556648 (+ 930us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:02.556652 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:02.556672 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:29:02.556758 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:02.556764 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:02.557963 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:02.557966 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:02.559781 (+ 1815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:02.559791 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:02.560890 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:02.560893 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:02.560989 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:02.561011 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 08:29:02.561061 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 08:29:02.561120 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":468,"thread_start_us":93,"threads_started":1}
I20260808 08:29:02.562879 22223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:02.555818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39660 (local address 127.22.235.195:34835)
0808 08:29:02.556220 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:02.556229 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:02.556248 (+ 19us) server_negotiation.cc:407] Connection header received
0808 08:29:02.556314 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:02.556320 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:02.556481 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:02.557011 (+ 530us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:29:02.557145 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:02.557840 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:02.559914 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:02.560761 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:02.561261 (+ 500us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:02.561406 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:02.561466 (+ 60us) server_negotiation.cc:299] Negotiation successful
0808 08:29:02.562744 (+ 1278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":117,"threads_started":1}
I20260808 08:29:02.564379 21894 tablet_service.cc:1467] Processing CreateTablet for tablet e82c148497704af6a76bf2a551409037 (DEFAULT_TABLE table=test-table [id=55bbcc9fb71b47368de1a0f6b7da1dde]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:29:02.564827 21694 tablet_service.cc:1467] Processing CreateTablet for tablet e82c148497704af6a76bf2a551409037 (DEFAULT_TABLE table=test-table [id=55bbcc9fb71b47368de1a0f6b7da1dde]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:29:02.565416 21894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e82c148497704af6a76bf2a551409037. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:29:02.565740 21694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e82c148497704af6a76bf2a551409037. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:29:02.571615 22225 tablet_bootstrap.cc:492] T e82c148497704af6a76bf2a551409037 P e353b4afab41427690334c8eadeb8d17: Bootstrap starting.
I20260808 08:29:02.571862 22097 tablet_service.cc:1467] Processing CreateTablet for tablet e82c148497704af6a76bf2a551409037 (DEFAULT_TABLE table=test-table [id=55bbcc9fb71b47368de1a0f6b7da1dde]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:29:02.573107 22226 tablet_bootstrap.cc:492] T e82c148497704af6a76bf2a551409037 P 3d1c33d59ab44ac6a9791e113978fec6: Bootstrap starting.
I20260808 08:29:02.573376 22097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e82c148497704af6a76bf2a551409037. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:29:02.574261 22225 tablet_bootstrap.cc:654] T e82c148497704af6a76bf2a551409037 P e353b4afab41427690334c8eadeb8d17: Neither blocks nor log segments found. Creating new log.
I20260808 08:29:02.575309 22226 tablet_bootstrap.cc:654] T e82c148497704af6a76bf2a551409037 P 3d1c33d59ab44ac6a9791e113978fec6: Neither blocks nor log segments found. Creating new log.
I20260808 08:29:02.575541 22225 log.cc:824] T e82c148497704af6a76bf2a551409037 P e353b4afab41427690334c8eadeb8d17: Log is configured to *not* fsync() on all Append() calls
I20260808 08:29:02.576431 22226 log.cc:824] T e82c148497704af6a76bf2a551409037 P 3d1c33d59ab44ac6a9791e113978fec6: Log is configured to *not* fsync() on all Append() calls
I20260808 08:29:02.577667 22225 tablet_bootstrap.cc:492] T e82c148497704af6a76bf2a551409037 P e353b4afab41427690334c8eadeb8d17: No bootstrap required, opened a new log
I20260808 08:29:02.577864 22225 ts_tablet_manager.cc:1397] T e82c148497704af6a76bf2a551409037 P e353b4afab41427690334c8eadeb8d17: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 08:29:02.578413 22226 tablet_bootstrap.cc:492] T e82c148497704af6a76bf2a551409037 P 3d1c33d59ab44ac6a9791e113978fec6: No bootstrap required, opened a new log
I20260808 08:29:02.578670 22226 ts_tablet_manager.cc:1397] T e82c148497704af6a76bf2a551409037 P 3d1c33d59ab44ac6a9791e113978fec6: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 08:29:02.580049 22230 tablet_bootstrap.cc:492] T e82c148497704af6a76bf2a551409037 P a916a34e157c446a9282d8a4533dba81: Bootstrap starting.
I20260808 08:29:02.581954 22230 tablet_bootstrap.cc:654] T e82c148497704af6a76bf2a551409037 P a916a34e157c446a9282d8a4533dba81: Neither blocks nor log segments found. Creating new log.
I20260808 08:29:02.581912 22225 raft_consensus.cc:348] T e82c148497704af6a76bf2a551409037 P e353b4afab41427690334c8eadeb8d17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d1c33d59ab44ac6a9791e113978fec6" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45581 } } peers { permanent_uuid: "e353b4afab41427690334c8eadeb8d17" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 40049 } } peers { permanent_uuid: "a916a34e157c446a9282d8a4533dba81" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34835 } }
I20260808 08:29:02.582193 22225 raft_consensus.cc:374] T e82c148497704af6a76bf2a551409037 P e353b4afab41427690334c8eadeb8d17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:29:02.582278 22225 raft_consensus.cc:729] T e82c148497704af6a76bf2a551409037 P e353b4afab41427690334c8eadeb8d17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e353b4afab41427690334c8eadeb8d17, State: Initialized, Role: FOLLOWER
I20260808 08:29:02.582376 22226 raft_consensus.cc:348] T e82c148497704af6a76bf2a551409037 P 3d1c33d59ab44ac6a9791e113978fec6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d1c33d59ab44ac6a9791e113978fec6" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45581 } } peers { permanent_uuid: "e353b4afab41427690334c8eadeb8d17" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 40049 } } peers { permanent_uuid: "a916a34e157c446a9282d8a4533dba81" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34835 } }
I20260808 08:29:02.582624 22226 raft_consensus.cc:374] T e82c148497704af6a76bf2a551409037 P 3d1c33d59ab44ac6a9791e113978fec6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:29:02.582715 22226 raft_consensus.cc:729] T e82c148497704af6a76bf2a551409037 P 3d1c33d59ab44ac6a9791e113978fec6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d1c33d59ab44ac6a9791e113978fec6, State: Initialized, Role: FOLLOWER
I20260808 08:29:02.582738 22225 consensus_queue.cc:260] T e82c148497704af6a76bf2a551409037 P e353b4afab41427690334c8eadeb8d17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d1c33d59ab44ac6a9791e113978fec6" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45581 } } peers { permanent_uuid: "e353b4afab41427690334c8eadeb8d17" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 40049 } } peers { permanent_uuid: "a916a34e157c446a9282d8a4533dba81" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34835 } }
I20260808 08:29:02.583130 22230 log.cc:824] T e82c148497704af6a76bf2a551409037 P a916a34e157c446a9282d8a4533dba81: Log is configured to *not* fsync() on all Append() calls
I20260808 08:29:02.583197 22226 consensus_queue.cc:260] T e82c148497704af6a76bf2a551409037 P 3d1c33d59ab44ac6a9791e113978fec6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d1c33d59ab44ac6a9791e113978fec6" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45581 } } peers { permanent_uuid: "e353b4afab41427690334c8eadeb8d17" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 40049 } } peers { permanent_uuid: "a916a34e157c446a9282d8a4533dba81" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34835 } }
I20260808 08:29:02.583436 21995 heartbeater.cc:503] Master 127.22.235.254:39449 was elected leader, sending a full tablet report...
I20260808 08:29:02.583745 22225 ts_tablet_manager.cc:1428] T e82c148497704af6a76bf2a551409037 P e353b4afab41427690334c8eadeb8d17: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 08:29:02.584160 22226 ts_tablet_manager.cc:1428] T e82c148497704af6a76bf2a551409037 P 3d1c33d59ab44ac6a9791e113978fec6: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 08:29:02.584268 21790 heartbeater.cc:503] Master 127.22.235.254:39449 was elected leader, sending a full tablet report...
I20260808 08:29:02.586606 22230 tablet_bootstrap.cc:492] T e82c148497704af6a76bf2a551409037 P a916a34e157c446a9282d8a4533dba81: No bootstrap required, opened a new log
I20260808 08:29:02.586856 22230 ts_tablet_manager.cc:1397] T e82c148497704af6a76bf2a551409037 P a916a34e157c446a9282d8a4533dba81: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 08:29:02.590016 22230 raft_consensus.cc:348] T e82c148497704af6a76bf2a551409037 P a916a34e157c446a9282d8a4533dba81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d1c33d59ab44ac6a9791e113978fec6" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45581 } } peers { permanent_uuid: "e353b4afab41427690334c8eadeb8d17" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 40049 } } peers { permanent_uuid: "a916a34e157c446a9282d8a4533dba81" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34835 } }
I20260808 08:29:02.590216 22230 raft_consensus.cc:374] T e82c148497704af6a76bf2a551409037 P a916a34e157c446a9282d8a4533dba81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:29:02.590255 22230 raft_consensus.cc:729] T e82c148497704af6a76bf2a551409037 P a916a34e157c446a9282d8a4533dba81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a916a34e157c446a9282d8a4533dba81, State: Initialized, Role: FOLLOWER
I20260808 08:29:02.590615 22230 consensus_queue.cc:260] T e82c148497704af6a76bf2a551409037 P a916a34e157c446a9282d8a4533dba81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d1c33d59ab44ac6a9791e113978fec6" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45581 } } peers { permanent_uuid: "e353b4afab41427690334c8eadeb8d17" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 40049 } } peers { permanent_uuid: "a916a34e157c446a9282d8a4533dba81" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34835 } }
I20260808 08:29:02.591241 22202 heartbeater.cc:503] Master 127.22.235.254:39449 was elected leader, sending a full tablet report...
I20260808 08:29:02.591499 22230 ts_tablet_manager.cc:1428] T e82c148497704af6a76bf2a551409037 P a916a34e157c446a9282d8a4533dba81: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
W20260808 08:29:02.764050 22203 tablet.cc:2369] T e82c148497704af6a76bf2a551409037 P a916a34e157c446a9282d8a4533dba81: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 08:29:02.784767 21997 tablet.cc:2369] T e82c148497704af6a76bf2a551409037 P e353b4afab41427690334c8eadeb8d17: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 08:29:02.823113 21791 tablet.cc:2369] T e82c148497704af6a76bf2a551409037 P 3d1c33d59ab44ac6a9791e113978fec6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 08:29:02.884711 22237 raft_consensus.cc:482] T e82c148497704af6a76bf2a551409037 P e353b4afab41427690334c8eadeb8d17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 08:29:02.884956 22237 raft_consensus.cc:504] T e82c148497704af6a76bf2a551409037 P e353b4afab41427690334c8eadeb8d17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d1c33d59ab44ac6a9791e113978fec6" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45581 } } peers { permanent_uuid: "e353b4afab41427690334c8eadeb8d17" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 40049 } } peers { permanent_uuid: "a916a34e157c446a9282d8a4533dba81" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34835 } }
I20260808 08:29:02.886111 22237 leader_election.cc:288] T e82c148497704af6a76bf2a551409037 P e353b4afab41427690334c8eadeb8d17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d1c33d59ab44ac6a9791e113978fec6 (127.22.235.193:45581), a916a34e157c446a9282d8a4533dba81 (127.22.235.195:34835)
I20260808 08:29:02.889978 22223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:02.886485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:36511 (local address 127.22.235.195:34835)
0808 08:29:02.886611 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:02.886615 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:02.886962 (+ 347us) server_negotiation.cc:407] Connection header received
0808 08:29:02.887073 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:02.887076 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:02.887123 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:02.887198 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:29:02.887452 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:02.887925 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:02.889063 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:02.889440 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:02.889539 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:02.889594 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:02.889771 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 08:29:02.889849 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 08:29:02.889983 22243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:02.886384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.195:34835 (local address 127.22.235.194:36511)
0808 08:29:02.886835 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:02.886849 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:02.886981 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:02.887211 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:02.887213 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:02.887228 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:29:02.887347 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:02.887353 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:02.888033 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:02.888036 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:02.888938 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:02.888947 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:02.889556 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:02.889560 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:02.889654 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:02.889679 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 08:29:02.889740 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 08:29:02.889809 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"mutex_wait_us":20,"thread_start_us":81,"threads_started":1}
I20260808 08:29:02.890017 22219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:02.886451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:57415 (local address 127.22.235.193:45581)
0808 08:29:02.886598 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:02.886603 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:02.886778 (+ 175us) server_negotiation.cc:407] Connection header received
0808 08:29:02.886918 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:02.886922 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:02.886980 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:02.887053 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:29:02.887344 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:02.887884 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:02.889081 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:02.889459 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:02.889580 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:02.889635 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:02.889837 (+ 202us) server_negotiation.cc:299] Negotiation successful
0808 08:29:02.889913 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 08:29:02.889997 22241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:02.886357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:45581 (local address 127.22.235.194:57415)
0808 08:29:02.886685 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:02.886697 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:02.886810 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:02.887070 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:02.887073 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:02.887092 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:29:02.887223 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:02.887231 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:02.887994 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:02.887998 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:02.888948 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:02.888957 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:02.889577 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:02.889581 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:02.889682 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:02.889706 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 08:29:02.889798 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 08:29:02.889861 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"mutex_wait_us":57,"thread_start_us":90,"threads_started":1}
I20260808 08:29:02.890604 22125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e82c148497704af6a76bf2a551409037" candidate_uuid: "e353b4afab41427690334c8eadeb8d17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a916a34e157c446a9282d8a4533dba81" is_pre_election: true
I20260808 08:29:02.890609 21720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e82c148497704af6a76bf2a551409037" candidate_uuid: "e353b4afab41427690334c8eadeb8d17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d1c33d59ab44ac6a9791e113978fec6" is_pre_election: true
I20260808 08:29:02.890877 21720 raft_consensus.cc:2454] T e82c148497704af6a76bf2a551409037 P 3d1c33d59ab44ac6a9791e113978fec6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e353b4afab41427690334c8eadeb8d17 in term 0.
I20260808 08:29:02.890882 22125 raft_consensus.cc:2454] T e82c148497704af6a76bf2a551409037 P a916a34e157c446a9282d8a4533dba81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e353b4afab41427690334c8eadeb8d17 in term 0.
I20260808 08:29:02.891343 21827 leader_election.cc:302] T e82c148497704af6a76bf2a551409037 P e353b4afab41427690334c8eadeb8d17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a916a34e157c446a9282d8a4533dba81, e353b4afab41427690334c8eadeb8d17; no voters:
I20260808 08:29:02.891631 22237 raft_consensus.cc:2781] T e82c148497704af6a76bf2a551409037 P e353b4afab41427690334c8eadeb8d17 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 08:29:02.891710 22237 raft_consensus.cc:482] T e82c148497704af6a76bf2a551409037 P e353b4afab41427690334c8eadeb8d17 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 08:29:02.891762 22237 raft_consensus.cc:3037] T e82c148497704af6a76bf2a551409037 P e353b4afab41427690334c8eadeb8d17 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:29:02.892833 22237 raft_consensus.cc:504] T e82c148497704af6a76bf2a551409037 P e353b4afab41427690334c8eadeb8d17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d1c33d59ab44ac6a9791e113978fec6" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45581 } } peers { permanent_uuid: "e353b4afab41427690334c8eadeb8d17" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 40049 } } peers { permanent_uuid: "a916a34e157c446a9282d8a4533dba81" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34835 } }
I20260808 08:29:02.893174 22237 leader_election.cc:288] T e82c148497704af6a76bf2a551409037 P e353b4afab41427690334c8eadeb8d17 [CANDIDATE]: Term 1 election: Requested vote from peers 3d1c33d59ab44ac6a9791e113978fec6 (127.22.235.193:45581), a916a34e157c446a9282d8a4533dba81 (127.22.235.195:34835)
I20260808 08:29:02.893517 21720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e82c148497704af6a76bf2a551409037" candidate_uuid: "e353b4afab41427690334c8eadeb8d17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d1c33d59ab44ac6a9791e113978fec6"
I20260808 08:29:02.893510 22125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e82c148497704af6a76bf2a551409037" candidate_uuid: "e353b4afab41427690334c8eadeb8d17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a916a34e157c446a9282d8a4533dba81"
I20260808 08:29:02.893631 21720 raft_consensus.cc:3037] T e82c148497704af6a76bf2a551409037 P 3d1c33d59ab44ac6a9791e113978fec6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:29:02.893631 22125 raft_consensus.cc:3037] T e82c148497704af6a76bf2a551409037 P a916a34e157c446a9282d8a4533dba81 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:29:02.894863 21720 raft_consensus.cc:2454] T e82c148497704af6a76bf2a551409037 P 3d1c33d59ab44ac6a9791e113978fec6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e353b4afab41427690334c8eadeb8d17 in term 1.
I20260808 08:29:02.894863 22125 raft_consensus.cc:2454] T e82c148497704af6a76bf2a551409037 P a916a34e157c446a9282d8a4533dba81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e353b4afab41427690334c8eadeb8d17 in term 1.
I20260808 08:29:02.895170 21822 leader_election.cc:302] T e82c148497704af6a76bf2a551409037 P e353b4afab41427690334c8eadeb8d17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d1c33d59ab44ac6a9791e113978fec6, e353b4afab41427690334c8eadeb8d17; no voters:
I20260808 08:29:02.895319 22237 raft_consensus.cc:2781] T e82c148497704af6a76bf2a551409037 P e353b4afab41427690334c8eadeb8d17 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:29:02.895538 22237 raft_consensus.cc:686] T e82c148497704af6a76bf2a551409037 P e353b4afab41427690334c8eadeb8d17 [term 1 LEADER]: Becoming Leader. State: Replica: e353b4afab41427690334c8eadeb8d17, State: Running, Role: LEADER
I20260808 08:29:02.895864 22237 consensus_queue.cc:237] T e82c148497704af6a76bf2a551409037 P e353b4afab41427690334c8eadeb8d17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d1c33d59ab44ac6a9791e113978fec6" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45581 } } peers { permanent_uuid: "e353b4afab41427690334c8eadeb8d17" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 40049 } } peers { permanent_uuid: "a916a34e157c446a9282d8a4533dba81" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34835 } }
I20260808 08:29:02.898931 21494 catalog_manager.cc:5495] T e82c148497704af6a76bf2a551409037 P e353b4afab41427690334c8eadeb8d17 reported cstate change: term changed from 0 to 1, leader changed from <none> to e353b4afab41427690334c8eadeb8d17 (127.22.235.194). New cstate: current_term: 1 leader_uuid: "e353b4afab41427690334c8eadeb8d17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d1c33d59ab44ac6a9791e113978fec6" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e353b4afab41427690334c8eadeb8d17" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 40049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a916a34e157c446a9282d8a4533dba81" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34835 } health_report { overall_health: UNKNOWN } } }
I20260808 08:29:02.930505 22222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:02.927149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55562 (local address 127.22.235.194:40049)
0808 08:29:02.927268 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:02.927272 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:02.927284 (+ 12us) server_negotiation.cc:407] Connection header received
0808 08:29:02.927351 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:02.927353 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:02.927406 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:02.927489 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:29:02.927704 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:02.928202 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:02.928982 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:02.929188 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:02.929255 (+ 67us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:02.929608 (+ 353us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:29:02.930083 (+ 475us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 08:29:02.930194 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:02.930323 (+ 129us) server_negotiation.cc:299] Negotiation successful
0808 08:29:02.930364 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 08:29:02.936362 22125 raft_consensus.cc:1261] T e82c148497704af6a76bf2a551409037 P a916a34e157c446a9282d8a4533dba81 [term 1 FOLLOWER]: Refusing update from remote peer e353b4afab41427690334c8eadeb8d17: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:29:02.936362 21720 raft_consensus.cc:1261] T e82c148497704af6a76bf2a551409037 P 3d1c33d59ab44ac6a9791e113978fec6 [term 1 FOLLOWER]: Refusing update from remote peer e353b4afab41427690334c8eadeb8d17: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:29:02.937037 22244 consensus_queue.cc:1035] T e82c148497704af6a76bf2a551409037 P e353b4afab41427690334c8eadeb8d17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a916a34e157c446a9282d8a4533dba81" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 34835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:29:02.937208 22237 consensus_queue.cc:1035] T e82c148497704af6a76bf2a551409037 P e353b4afab41427690334c8eadeb8d17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d1c33d59ab44ac6a9791e113978fec6" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:29:02.946610 22251 mvcc.cc:205] Tried to move back new op lower bound from 7316184035060436992 to 7316184034903998464. Current Snapshot: MvccSnapshot[applied={T|T < 7316184035060436992}]
I20260808 08:29:02.953207 21494 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49160:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:29:02.953398 21494 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49160:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:29:02.955828 21494 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cdbafa9845a4418db97b983bc6f3e4c7: Sending DeleteTablet for 3 replicas of tablet e82c148497704af6a76bf2a551409037
I20260808 08:29:02.956601 21694 tablet_service.cc:1514] Processing DeleteTablet for tablet e82c148497704af6a76bf2a551409037 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:29:02 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:33944
I20260808 08:29:02.956851 22097 tablet_service.cc:1514] Processing DeleteTablet for tablet e82c148497704af6a76bf2a551409037 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:29:02 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:39660
I20260808 08:29:02.957187 21894 tablet_service.cc:1514] Processing DeleteTablet for tablet e82c148497704af6a76bf2a551409037 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:29:02 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:55560
I20260808 08:29:02.957363 22270 tablet_replica.cc:330] T e82c148497704af6a76bf2a551409037 P a916a34e157c446a9282d8a4533dba81: stopping tablet replica
I20260808 08:29:02.957618 22271 tablet_replica.cc:330] T e82c148497704af6a76bf2a551409037 P e353b4afab41427690334c8eadeb8d17: stopping tablet replica
I20260808 08:29:02.957712 22270 raft_consensus.cc:2229] T e82c148497704af6a76bf2a551409037 P a916a34e157c446a9282d8a4533dba81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 08:29:02.957988 22271 raft_consensus.cc:2229] T e82c148497704af6a76bf2a551409037 P e353b4afab41427690334c8eadeb8d17 [term 1 LEADER]: Raft consensus shutting down.
I20260808 08:29:02.958096 22270 raft_consensus.cc:2258] T e82c148497704af6a76bf2a551409037 P a916a34e157c446a9282d8a4533dba81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:29:02.958346 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 21597
I20260808 08:29:02.958418 22271 raft_consensus.cc:2258] T e82c148497704af6a76bf2a551409037 P e353b4afab41427690334c8eadeb8d17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:29:02.960198 22270 ts_tablet_manager.cc:1905] T e82c148497704af6a76bf2a551409037 P a916a34e157c446a9282d8a4533dba81: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:29:02.962742 22271 ts_tablet_manager.cc:1905] T e82c148497704af6a76bf2a551409037 P e353b4afab41427690334c8eadeb8d17: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:29:02.963296 22270 ts_tablet_manager.cc:1918] T e82c148497704af6a76bf2a551409037 P a916a34e157c446a9282d8a4533dba81: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 08:29:02.963502 22270 log.cc:1196] T e82c148497704af6a76bf2a551409037 P a916a34e157c446a9282d8a4533dba81: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-2/wal/wals/e82c148497704af6a76bf2a551409037
I20260808 08:29:02.963948 22270 ts_tablet_manager.cc:1939] T e82c148497704af6a76bf2a551409037 P a916a34e157c446a9282d8a4533dba81: Deleting consensus metadata
I20260808 08:29:02.965380 22271 ts_tablet_manager.cc:1918] T e82c148497704af6a76bf2a551409037 P e353b4afab41427690334c8eadeb8d17: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 08:29:02.965551 22271 log.cc:1196] T e82c148497704af6a76bf2a551409037 P e353b4afab41427690334c8eadeb8d17: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786177493204855-23471-0/minicluster-data/ts-1/wal/wals/e82c148497704af6a76bf2a551409037
I20260808 08:29:02.965585 21476 catalog_manager.cc:4841] TS a916a34e157c446a9282d8a4533dba81 (127.22.235.195:34835): tablet e82c148497704af6a76bf2a551409037 (table test-table [id=55bbcc9fb71b47368de1a0f6b7da1dde]) successfully deleted
I20260808 08:29:02.965957 22271 ts_tablet_manager.cc:1939] T e82c148497704af6a76bf2a551409037 P e353b4afab41427690334c8eadeb8d17: Deleting consensus metadata
I20260808 08:29:02.967075 21476 catalog_manager.cc:4841] TS e353b4afab41427690334c8eadeb8d17 (127.22.235.194:40049): tablet e82c148497704af6a76bf2a551409037 (table test-table [id=55bbcc9fb71b47368de1a0f6b7da1dde]) successfully deleted
W20260808 08:29:02.974963 21468 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.235.193:45581 (error 108)
I20260808 08:29:02.975207 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 21797
I20260808 08:29:02.976608 22221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:02.976244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:45581 (local address 127.0.0.1:33960)
0808 08:29:02.976409 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:02.976504 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.193:45581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 08:29:02.976985 21468 catalog_manager.cc:4584] TS 3d1c33d59ab44ac6a9791e113978fec6 (127.22.235.193:45581): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e82c148497704af6a76bf2a551409037: Network error: Client connection negotiation failed: client connection to 127.22.235.193:45581: connect: Connection refused (error 111)
I20260808 08:29:02.983955 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 22001
I20260808 08:29:03.003979 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 21441
2026-08-08T08:29:03Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2703 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-08T08:29:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:29:03Z Disabled control of system clock
I20260808 08:29:03.042615 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:35389
--webserver_interface=127.22.235.254
--webserver_port=0
--builtin_ntp_servers=127.22.235.212:45245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:35389
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 08:29:03.155043 22283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:29:03.155296 22283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:29:03.155328 22283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:29:03.159283 22283 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:29:03.159337 22283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:29:03.159351 22283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:29:03.159368 22283 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:29:03.159396 22283 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:29:03.164125 22283 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:45245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:35389
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:35389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.235.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:29:03.165231 22283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:29:03.166672 22283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:29:03.175320 22291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:03.175376 22290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:03.175396 22283 server_base.cc:1034] running on GCE node
W20260808 08:29:03.175320 22294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:03.176054 22283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:29:03.177196 22283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:29:03.178354 22283 hybrid_clock.cc:648] HybridClock initialized: now 1786177743178332 us; error 37 us; skew 500 ppm
I20260808 08:29:03.180328 22283 webserver.cc:460] Webserver started at http://127.22.235.254:40227/ using document root <none> and password file <none>
I20260808 08:29:03.180946 22283 fs_manager.cc:359] Metadata directory not provided
I20260808 08:29:03.181012 22283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:29:03.181209 22283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:29:03.183239 22283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "87fac91252d2486f89e9edbb8cfcac45"
format_stamp: "Formatted at 2026-08-08 08:29:03 on dist-test-slave-f7th"
server_key: "d43b4e637ddd5346c6eeb36f72524123"
server_key_iv: "d4932fd490d29f2bca05635137ecdd03"
server_key_version: "encryptionkey@0"
I20260808 08:29:03.183887 22283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "87fac91252d2486f89e9edbb8cfcac45"
format_stamp: "Formatted at 2026-08-08 08:29:03 on dist-test-slave-f7th"
server_key: "d43b4e637ddd5346c6eeb36f72524123"
server_key_iv: "d4932fd490d29f2bca05635137ecdd03"
server_key_version: "encryptionkey@0"
I20260808 08:29:03.187733 22283 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260808 08:29:03.190181 22302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:03.191340 22283 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 08:29:03.191445 22283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "87fac91252d2486f89e9edbb8cfcac45"
format_stamp: "Formatted at 2026-08-08 08:29:03 on dist-test-slave-f7th"
server_key: "d43b4e637ddd5346c6eeb36f72524123"
server_key_iv: "d4932fd490d29f2bca05635137ecdd03"
server_key_version: "encryptionkey@0"
I20260808 08:29:03.191552 22283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:29:03.209570 22283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:29:03.210246 22283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:29:03.210420 22283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:29:03.217106 22283 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.254:35389
I20260808 08:29:03.217129 22391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.254:35389 every 8 connection(s)
I20260808 08:29:03.218051 22283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:29:03.221263 22392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:29:03.226765 22392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87fac91252d2486f89e9edbb8cfcac45: Bootstrap starting.
I20260808 08:29:03.228199 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 22283
I20260808 08:29:03.228312 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:29:03.228566 23471 external_mini_cluster.cc:1428] Setting key fe11644957f7796cecc4994558786b09
I20260808 08:29:03.229426 22392 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87fac91252d2486f89e9edbb8cfcac45: Neither blocks nor log segments found. Creating new log.
I20260808 08:29:03.230310 22392 log.cc:824] T 00000000000000000000000000000000 P 87fac91252d2486f89e9edbb8cfcac45: Log is configured to *not* fsync() on all Append() calls
I20260808 08:29:03.232582 22392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87fac91252d2486f89e9edbb8cfcac45: No bootstrap required, opened a new log
I20260808 08:29:03.236761 22392 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87fac91252d2486f89e9edbb8cfcac45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87fac91252d2486f89e9edbb8cfcac45" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 35389 } }
I20260808 08:29:03.237033 22392 raft_consensus.cc:374] T 00000000000000000000000000000000 P 87fac91252d2486f89e9edbb8cfcac45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:29:03.237125 22392 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87fac91252d2486f89e9edbb8cfcac45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87fac91252d2486f89e9edbb8cfcac45, State: Initialized, Role: FOLLOWER
I20260808 08:29:03.237653 22392 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87fac91252d2486f89e9edbb8cfcac45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87fac91252d2486f89e9edbb8cfcac45" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 35389 } }
I20260808 08:29:03.237818 22392 raft_consensus.cc:388] T 00000000000000000000000000000000 P 87fac91252d2486f89e9edbb8cfcac45 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:29:03.237893 22392 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87fac91252d2486f89e9edbb8cfcac45 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:29:03.237987 22392 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87fac91252d2486f89e9edbb8cfcac45 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:29:03.238471 22399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:03.229699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33104 (local address 127.22.235.254:35389)
0808 08:29:03.230416 (+ 717us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:03.230436 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:03.230480 (+ 44us) server_negotiation.cc:407] Connection header received
0808 08:29:03.231197 (+ 717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:03.231220 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:03.231634 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:03.232001 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:03.232425 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:03.233893 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:03.234746 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:03.235452 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:03.235633 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:03.236352 (+ 719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:03.236359 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:03.236375 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:29:03.236882 (+ 507us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:03.236953 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:29:03.236993 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:03.237099 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:03.237240 (+ 141us) server_negotiation.cc:299] Negotiation successful
0808 08:29:03.237507 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":122,"threads_started":1}
I20260808 08:29:03.239014 22392 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87fac91252d2486f89e9edbb8cfcac45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87fac91252d2486f89e9edbb8cfcac45" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 35389 } }
I20260808 08:29:03.239331 22392 leader_election.cc:302] T 00000000000000000000000000000000 P 87fac91252d2486f89e9edbb8cfcac45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 87fac91252d2486f89e9edbb8cfcac45; no voters:
I20260808 08:29:03.239677 22392 leader_election.cc:288] T 00000000000000000000000000000000 P 87fac91252d2486f89e9edbb8cfcac45 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:29:03.239732 22404 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87fac91252d2486f89e9edbb8cfcac45 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:29:03.239933 22404 raft_consensus.cc:686] T 00000000000000000000000000000000 P 87fac91252d2486f89e9edbb8cfcac45 [term 1 LEADER]: Becoming Leader. State: Replica: 87fac91252d2486f89e9edbb8cfcac45, State: Running, Role: LEADER
I20260808 08:29:03.240291 22404 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87fac91252d2486f89e9edbb8cfcac45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87fac91252d2486f89e9edbb8cfcac45" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 35389 } }
I20260808 08:29:03.241096 22392 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87fac91252d2486f89e9edbb8cfcac45 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:29:03.242091 22405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87fac91252d2486f89e9edbb8cfcac45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "87fac91252d2486f89e9edbb8cfcac45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87fac91252d2486f89e9edbb8cfcac45" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 35389 } } }
I20260808 08:29:03.242226 22405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87fac91252d2486f89e9edbb8cfcac45 [sys.catalog]: This master's current role is: LEADER
I20260808 08:29:03.242599 22417 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:29:03.242564 22406 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87fac91252d2486f89e9edbb8cfcac45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87fac91252d2486f89e9edbb8cfcac45. Latest consensus state: current_term: 1 leader_uuid: "87fac91252d2486f89e9edbb8cfcac45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87fac91252d2486f89e9edbb8cfcac45" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 35389 } } }
I20260808 08:29:03.242697 22406 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87fac91252d2486f89e9edbb8cfcac45 [sys.catalog]: This master's current role is: LEADER
I20260808 08:29:03.245687 22417 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:29:03.253039 22417 catalog_manager.cc:1347] Generated new cluster ID: 07e699224a104c2f9d6d625ba009a86d
I20260808 08:29:03.253110 22417 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:29:03.270964 22417 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:29:03.271878 22417 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:29:03.279539 22417 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 87fac91252d2486f89e9edbb8cfcac45: Generated new TSK 0
I20260808 08:29:03.280196 22417 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 08:29:03.282917 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.193:0
--local_ip_for_outbound_sockets=127.22.235.193
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:35389
--builtin_ntp_servers=127.22.235.212:45245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 08:29:03.395046 22434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:29:03.395313 22434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:29:03.395350 22434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:29:03.399295 22434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:29:03.399349 22434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:29:03.399401 22434 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.193
I20260808 08:29:03.403965 22434 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:45245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:35389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.193
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:29:03.404953 22434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:29:03.406252 22434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:29:03.415897 22440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:03.415921 22444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:03.415894 22442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:03.416193 22434 server_base.cc:1034] running on GCE node
I20260808 08:29:03.416575 22434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:29:03.417194 22434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:29:03.418365 22434 hybrid_clock.cc:648] HybridClock initialized: now 1786177743418342 us; error 37 us; skew 500 ppm
I20260808 08:29:03.420101 22434 webserver.cc:460] Webserver started at http://127.22.235.193:43645/ using document root <none> and password file <none>
I20260808 08:29:03.420680 22434 fs_manager.cc:359] Metadata directory not provided
I20260808 08:29:03.420751 22434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:29:03.420956 22434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:29:03.422936 22434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-0/data/instance:
uuid: "fe7ca79af5664fba8c260aee0614d2d4"
format_stamp: "Formatted at 2026-08-08 08:29:03 on dist-test-slave-f7th"
server_key: "7c0c2cff106a6745fb1e863ebc6b2e33"
server_key_iv: "b1c48712cc8460062050d797ce2851b4"
server_key_version: "encryptionkey@0"
I20260808 08:29:03.423460 22434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance:
uuid: "fe7ca79af5664fba8c260aee0614d2d4"
format_stamp: "Formatted at 2026-08-08 08:29:03 on dist-test-slave-f7th"
server_key: "7c0c2cff106a6745fb1e863ebc6b2e33"
server_key_iv: "b1c48712cc8460062050d797ce2851b4"
server_key_version: "encryptionkey@0"
I20260808 08:29:03.426939 22434 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 08:29:03.429219 22457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:03.430364 22434 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:29:03.430470 22434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-0/wal
uuid: "fe7ca79af5664fba8c260aee0614d2d4"
format_stamp: "Formatted at 2026-08-08 08:29:03 on dist-test-slave-f7th"
server_key: "7c0c2cff106a6745fb1e863ebc6b2e33"
server_key_iv: "b1c48712cc8460062050d797ce2851b4"
server_key_version: "encryptionkey@0"
I20260808 08:29:03.430557 22434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:29:03.451026 22434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:29:03.451673 22434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:29:03.451843 22434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:29:03.452407 22434 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:29:03.453354 22434 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:29:03.453402 22434 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:03.453455 22434 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:29:03.453483 22434 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:03.461740 22434 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.193:37169
I20260808 08:29:03.461800 22638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.193:37169 every 8 connection(s)
I20260808 08:29:03.462627 22434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
I20260808 08:29:03.470136 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 22434
I20260808 08:29:03.470244 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance
I20260808 08:29:03.470503 23471 external_mini_cluster.cc:1428] Setting key 562606d53a404d6fd134ac1496410419
I20260808 08:29:03.471709 22399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:03.464379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:38717 (local address 127.22.235.254:35389)
0808 08:29:03.464512 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:03.464515 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:03.465619 (+ 1104us) server_negotiation.cc:407] Connection header received
0808 08:29:03.467033 (+ 1414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:03.467036 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:03.467098 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:03.467173 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:03.468078 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:03.468639 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:03.469757 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:03.469986 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:03.470076 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:03.471275 (+ 1199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:03.471279 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:03.471282 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:29:03.471295 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:03.471316 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:29:03.471325 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:03.471388 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:03.471546 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 08:29:03.471597 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 08:29:03.472538 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.194:0
--local_ip_for_outbound_sockets=127.22.235.194
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:35389
--builtin_ntp_servers=127.22.235.212:45245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 08:29:03.472589 22644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:03.464796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:35389 (local address 127.22.235.193:38717)
0808 08:29:03.465452 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:03.465506 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:03.466800 (+ 1294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:03.467359 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:03.467366 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:03.467451 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:29:03.467931 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:03.467944 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:03.468764 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:03.468767 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:03.469633 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:03.469643 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:03.470131 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:03.470135 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:03.470333 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:03.471004 (+ 671us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 08:29:03.471006 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:29:03.471060 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 08:29:03.471143 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:29:03.471405 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:29:03.471414 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:29:03.471419 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 08:29:03.471543 (+ 124us) client_negotiation.cc:240] Negotiation successful
0808 08:29:03.471815 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":110,"threads_started":1}
I20260808 08:29:03.473807 22639 heartbeater.cc:347] Connected to a master server at 127.22.235.254:35389
I20260808 08:29:03.474025 22639 heartbeater.cc:464] Registering TS with master...
I20260808 08:29:03.474521 22639 heartbeater.cc:511] Master 127.22.235.254:35389 requested a full tablet report, sending...
I20260808 08:29:03.476046 22327 ts_manager.cc:194] Registered new tserver with Master: fe7ca79af5664fba8c260aee0614d2d4 (127.22.235.193:37169)
I20260808 08:29:03.477596 22327 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.235.193:38717
W20260808 08:29:03.587152 22645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:29:03.587406 22645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:29:03.587442 22645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:29:03.591323 22645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:29:03.591375 22645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:29:03.591429 22645 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.194
I20260808 08:29:03.596141 22645 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:45245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:35389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.194
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:29:03.597177 22645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:29:03.598491 22645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:29:03.607782 22657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:03.607824 22645 server_base.cc:1034] running on GCE node
W20260808 08:29:03.607992 22656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:03.607832 22660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:03.608487 22645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:29:03.609117 22645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:29:03.610276 22645 hybrid_clock.cc:648] HybridClock initialized: now 1786177743610245 us; error 41 us; skew 500 ppm
I20260808 08:29:03.611959 22645 webserver.cc:460] Webserver started at http://127.22.235.194:35865/ using document root <none> and password file <none>
I20260808 08:29:03.612510 22645 fs_manager.cc:359] Metadata directory not provided
I20260808 08:29:03.612577 22645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:29:03.612797 22645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:29:03.614737 22645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-1/data/instance:
uuid: "4f9e7d5cc8ff480e95c5bd2c8aa9bb30"
format_stamp: "Formatted at 2026-08-08 08:29:03 on dist-test-slave-f7th"
server_key: "2229a6bfcdb5d26090b2271e0aad958b"
server_key_iv: "cc4c1581c897fa4f45c6cab59a782dd3"
server_key_version: "encryptionkey@0"
I20260808 08:29:03.615255 22645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance:
uuid: "4f9e7d5cc8ff480e95c5bd2c8aa9bb30"
format_stamp: "Formatted at 2026-08-08 08:29:03 on dist-test-slave-f7th"
server_key: "2229a6bfcdb5d26090b2271e0aad958b"
server_key_iv: "cc4c1581c897fa4f45c6cab59a782dd3"
server_key_version: "encryptionkey@0"
I20260808 08:29:03.618777 22645 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260808 08:29:03.621203 22667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:03.622407 22645 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:29:03.622515 22645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-1/wal
uuid: "4f9e7d5cc8ff480e95c5bd2c8aa9bb30"
format_stamp: "Formatted at 2026-08-08 08:29:03 on dist-test-slave-f7th"
server_key: "2229a6bfcdb5d26090b2271e0aad958b"
server_key_iv: "cc4c1581c897fa4f45c6cab59a782dd3"
server_key_version: "encryptionkey@0"
I20260808 08:29:03.622604 22645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:29:03.642319 22645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:29:03.643018 22645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:29:03.643188 22645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:29:03.643761 22645 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:29:03.644734 22645 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:29:03.644783 22645 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:03.644837 22645 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:29:03.644866 22645 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:03.653635 22645 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.194:33551
I20260808 08:29:03.653708 22860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.194:33551 every 8 connection(s)
I20260808 08:29:03.654539 22645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
I20260808 08:29:03.660174 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 22645
I20260808 08:29:03.660296 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance
I20260808 08:29:03.660660 23471 external_mini_cluster.cc:1428] Setting key 08038c95e79ff84aba980d342087bfa1
I20260808 08:29:03.662894 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.195:0
--local_ip_for_outbound_sockets=127.22.235.195
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:35389
--builtin_ntp_servers=127.22.235.212:45245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 08:29:03.664299 22399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:03.656331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:50961 (local address 127.22.235.254:35389)
0808 08:29:03.656477 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:03.656482 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:03.657598 (+ 1116us) server_negotiation.cc:407] Connection header received
0808 08:29:03.658983 (+ 1385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:03.658986 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:03.659035 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:03.659111 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:03.659988 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:03.660761 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:03.661973 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:03.662286 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:03.662380 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:03.663728 (+ 1348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:03.663733 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:03.663735 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:29:03.663752 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:03.663780 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:29:03.663790 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:03.663867 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:03.664091 (+ 224us) server_negotiation.cc:299] Negotiation successful
0808 08:29:03.664155 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 08:29:03.665421 22864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:03.656826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:35389 (local address 127.22.235.194:50961)
0808 08:29:03.657455 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:03.657497 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:03.658769 (+ 1272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:03.659267 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:03.659273 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:03.659355 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:29:03.659827 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:03.659839 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:03.660892 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:03.660895 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:03.661866 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:03.661877 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:03.662443 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:03.662448 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:03.662658 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:03.663368 (+ 710us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 08:29:03.663370 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:29:03.663424 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 08:29:03.663520 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:29:03.663901 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:29:03.663912 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:29:03.663919 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 08:29:03.664068 (+ 149us) client_negotiation.cc:240] Negotiation successful
0808 08:29:03.664348 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":104,"threads_started":1}
I20260808 08:29:03.667804 22861 heartbeater.cc:347] Connected to a master server at 127.22.235.254:35389
I20260808 08:29:03.668061 22861 heartbeater.cc:464] Registering TS with master...
I20260808 08:29:03.668694 22861 heartbeater.cc:511] Master 127.22.235.254:35389 requested a full tablet report, sending...
I20260808 08:29:03.672185 22327 ts_manager.cc:194] Registered new tserver with Master: 4f9e7d5cc8ff480e95c5bd2c8aa9bb30 (127.22.235.194:33551)
I20260808 08:29:03.672996 22327 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.235.194:50961
W20260808 08:29:03.783504 22866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:29:03.783768 22866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:29:03.783823 22866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:29:03.787842 22866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:29:03.787905 22866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:29:03.787981 22866 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.195
I20260808 08:29:03.792860 22866 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:45245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:35389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.195
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:29:03.793937 22866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:29:03.795383 22866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:29:03.804955 22884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:03.805194 22881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:03.805027 22888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:03.806427 22866 server_base.cc:1034] running on GCE node
I20260808 08:29:03.806838 22866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:29:03.807482 22866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:29:03.808662 22866 hybrid_clock.cc:648] HybridClock initialized: now 1786177743808639 us; error 33 us; skew 500 ppm
I20260808 08:29:03.810447 22866 webserver.cc:460] Webserver started at http://127.22.235.195:41695/ using document root <none> and password file <none>
I20260808 08:29:03.811013 22866 fs_manager.cc:359] Metadata directory not provided
I20260808 08:29:03.811075 22866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:29:03.811270 22866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:29:03.813695 22866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-2/data/instance:
uuid: "41f1798b08854d128ff96c564a91e3e0"
format_stamp: "Formatted at 2026-08-08 08:29:03 on dist-test-slave-f7th"
server_key: "4f555851f2a2cf9143dd9725d1065b89"
server_key_iv: "6f3d9657640579888052d26550cc7e47"
server_key_version: "encryptionkey@0"
I20260808 08:29:03.814303 22866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance:
uuid: "41f1798b08854d128ff96c564a91e3e0"
format_stamp: "Formatted at 2026-08-08 08:29:03 on dist-test-slave-f7th"
server_key: "4f555851f2a2cf9143dd9725d1065b89"
server_key_iv: "6f3d9657640579888052d26550cc7e47"
server_key_version: "encryptionkey@0"
I20260808 08:29:03.818312 22866 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 08:29:03.820868 22901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:03.822134 22866 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:29:03.822244 22866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-2/wal
uuid: "41f1798b08854d128ff96c564a91e3e0"
format_stamp: "Formatted at 2026-08-08 08:29:03 on dist-test-slave-f7th"
server_key: "4f555851f2a2cf9143dd9725d1065b89"
server_key_iv: "6f3d9657640579888052d26550cc7e47"
server_key_version: "encryptionkey@0"
I20260808 08:29:03.822337 22866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:29:03.849603 22866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:29:03.850423 22866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:29:03.850615 22866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:29:03.851250 22866 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:29:03.852283 22866 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:29:03.852391 22866 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:03.852458 22866 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:29:03.852490 22866 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:03.864441 22866 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.195:44729
I20260808 08:29:03.864488 23080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.195:44729 every 8 connection(s)
I20260808 08:29:03.865465 22866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
I20260808 08:29:03.872895 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 22866
I20260808 08:29:03.873013 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance
I20260808 08:29:03.873286 23471 external_mini_cluster.cc:1428] Setting key 657f727bd888e5bb69f7bd0ffb2c71a3
I20260808 08:29:03.877094 22399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:03.867593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:40525 (local address 127.22.235.254:35389)
0808 08:29:03.867779 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:03.867784 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:03.869238 (+ 1454us) server_negotiation.cc:407] Connection header received
0808 08:29:03.871148 (+ 1910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:03.871152 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:03.871203 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:03.871281 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:03.872557 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:03.873366 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:03.875069 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:03.875322 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:03.875402 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:03.876648 (+ 1246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:03.876653 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:03.876656 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:29:03.876671 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:03.876693 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:29:03.876703 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:03.876760 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:03.876930 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 08:29:03.876981 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 08:29:03.878225 23083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:03.868157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:35389 (local address 127.22.235.195:40525)
0808 08:29:03.869048 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:03.869109 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:03.870885 (+ 1776us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:03.871513 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:03.871522 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:03.871640 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:29:03.872387 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:03.872408 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:03.873502 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:03.873506 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:03.874940 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:03.874953 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:03.875468 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:03.875472 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:03.875680 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:03.876310 (+ 630us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 08:29:03.876312 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:29:03.876380 (+ 68us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 08:29:03.876466 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:29:03.876779 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:29:03.876787 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:29:03.876793 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 08:29:03.876930 (+ 137us) client_negotiation.cc:240] Negotiation successful
0808 08:29:03.877234 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":150,"threads_started":1}
I20260808 08:29:03.879312 23081 heartbeater.cc:347] Connected to a master server at 127.22.235.254:35389
I20260808 08:29:03.879578 23081 heartbeater.cc:464] Registering TS with master...
I20260808 08:29:03.880199 23081 heartbeater.cc:511] Master 127.22.235.254:35389 requested a full tablet report, sending...
I20260808 08:29:03.881456 22327 ts_manager.cc:194] Registered new tserver with Master: 41f1798b08854d128ff96c564a91e3e0 (127.22.235.195:44729)
I20260808 08:29:03.882018 22327 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.235.195:40525
I20260808 08:29:03.887307 23471 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 08:29:03.893149 22399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:03.889291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33112 (local address 127.22.235.254:35389)
0808 08:29:03.889403 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:03.889407 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:03.889614 (+ 207us) server_negotiation.cc:407] Connection header received
0808 08:29:03.889721 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:03.889724 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:03.889768 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:03.889834 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:03.890122 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:03.890629 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:03.891500 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:03.891719 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:03.891806 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:03.892682 (+ 876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:03.892686 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:03.892689 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:29:03.892703 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:03.892720 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:29:03.892728 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:03.892835 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:03.892976 (+ 141us) server_negotiation.cc:299] Negotiation successful
0808 08:29:03.893015 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 08:29:03.895990 22327 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33112:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 08:29:03.898988 22327 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 08:29:03.913074 23107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:03.907529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37568 (local address 127.22.235.193:37169)
0808 08:29:03.908128 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:03.908139 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:03.908456 (+ 317us) server_negotiation.cc:407] Connection header received
0808 08:29:03.908548 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:03.908553 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:03.908784 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:03.908931 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:29:03.909330 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:03.910215 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:03.911816 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:03.912359 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:03.912481 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:03.912704 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:03.912808 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 08:29:03.912906 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":92,"threads_started":1}
I20260808 08:29:03.913071 23105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:03.907330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:37169 (local address 127.0.0.1:37568)
0808 08:29:03.908167 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:03.908180 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:03.908323 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:03.909074 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:03.909079 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:03.909106 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:29:03.909214 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:03.909219 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:03.910313 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:03.910316 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:03.911671 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:03.911683 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:03.912477 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:03.912481 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:03.912596 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:03.912675 (+ 79us) client_negotiation.cc:769] Sending connection context
0808 08:29:03.912784 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 08:29:03.912906 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":773,"mutex_wait_us":29,"spinlock_wait_cycles":5120,"thread_start_us":134,"threads_started":1}
I20260808 08:29:03.913733 23106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:03.907343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42250 (local address 127.22.235.194:33551)
0808 08:29:03.908023 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:03.908036 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:03.908275 (+ 239us) server_negotiation.cc:407] Connection header received
0808 08:29:03.908525 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:03.908532 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:03.908771 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:03.908924 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:29:03.909360 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:03.910495 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:03.911868 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:03.912665 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:03.912772 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:03.912924 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:03.913488 (+ 564us) server_negotiation.cc:299] Negotiation successful
0808 08:29:03.913579 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":110,"threads_started":1}
I20260808 08:29:03.913390 23104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:03.907227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.194:33551 (local address 127.0.0.1:42250)
0808 08:29:03.908120 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:03.908168 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:03.908355 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:03.909073 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:03.909082 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:03.909108 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:29:03.909222 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:03.909229 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:03.910611 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:03.910615 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:03.911757 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:03.911766 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:03.912987 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:03.912990 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:03.913074 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:03.913097 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 08:29:03.913166 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 08:29:03.913235 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":715,"mutex_wait_us":18,"spinlock_wait_cycles":15744,"thread_start_us":81,"threads_started":1}
I20260808 08:29:03.913889 23109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:03.908496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59010 (local address 127.22.235.195:44729)
0808 08:29:03.909286 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:03.909299 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:03.909320 (+ 21us) server_negotiation.cc:407] Connection header received
0808 08:29:03.909385 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:03.909390 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:03.909557 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:03.909698 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:29:03.909931 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:03.910878 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:03.912548 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:03.913305 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:03.913419 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:03.913575 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:03.913655 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 08:29:03.913734 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":690,"thread_start_us":127,"threads_started":1}
I20260808 08:29:03.915182 23108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:03.907954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.195:44729 (local address 127.0.0.1:59010)
0808 08:29:03.908970 (+ 1016us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:03.908982 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:03.909057 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:03.909694 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:03.909697 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:03.909717 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:29:03.909791 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:03.909796 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:03.911002 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:03.911005 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:03.912407 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:03.912418 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:03.913423 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:03.913427 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:03.913524 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:03.913548 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 08:29:03.913628 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 08:29:03.913693 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":956,"thread_start_us":113,"threads_started":1}
I20260808 08:29:03.916656 22533 tablet_service.cc:1467] Processing CreateTablet for tablet 4754eae416d54113a371e6a728514376 (DEFAULT_TABLE table=test-table [id=133bbbc523d24c559d057e31543e30e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:29:03.917158 22752 tablet_service.cc:1467] Processing CreateTablet for tablet 4754eae416d54113a371e6a728514376 (DEFAULT_TABLE table=test-table [id=133bbbc523d24c559d057e31543e30e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:29:03.917740 22533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4754eae416d54113a371e6a728514376. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:29:03.918210 22752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4754eae416d54113a371e6a728514376. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:29:03.917810 22976 tablet_service.cc:1467] Processing CreateTablet for tablet 4754eae416d54113a371e6a728514376 (DEFAULT_TABLE table=test-table [id=133bbbc523d24c559d057e31543e30e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:29:03.918784 22976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4754eae416d54113a371e6a728514376. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:29:03.923825 23110 tablet_bootstrap.cc:492] T 4754eae416d54113a371e6a728514376 P fe7ca79af5664fba8c260aee0614d2d4: Bootstrap starting.
I20260808 08:29:03.925271 23111 tablet_bootstrap.cc:492] T 4754eae416d54113a371e6a728514376 P 4f9e7d5cc8ff480e95c5bd2c8aa9bb30: Bootstrap starting.
I20260808 08:29:03.927245 23110 tablet_bootstrap.cc:654] T 4754eae416d54113a371e6a728514376 P fe7ca79af5664fba8c260aee0614d2d4: Neither blocks nor log segments found. Creating new log.
I20260808 08:29:03.927371 23111 tablet_bootstrap.cc:654] T 4754eae416d54113a371e6a728514376 P 4f9e7d5cc8ff480e95c5bd2c8aa9bb30: Neither blocks nor log segments found. Creating new log.
I20260808 08:29:03.928153 23111 log.cc:824] T 4754eae416d54113a371e6a728514376 P 4f9e7d5cc8ff480e95c5bd2c8aa9bb30: Log is configured to *not* fsync() on all Append() calls
I20260808 08:29:03.928153 23110 log.cc:824] T 4754eae416d54113a371e6a728514376 P fe7ca79af5664fba8c260aee0614d2d4: Log is configured to *not* fsync() on all Append() calls
I20260808 08:29:03.928886 23112 tablet_bootstrap.cc:492] T 4754eae416d54113a371e6a728514376 P 41f1798b08854d128ff96c564a91e3e0: Bootstrap starting.
I20260808 08:29:03.930248 23111 tablet_bootstrap.cc:492] T 4754eae416d54113a371e6a728514376 P 4f9e7d5cc8ff480e95c5bd2c8aa9bb30: No bootstrap required, opened a new log
I20260808 08:29:03.930363 23110 tablet_bootstrap.cc:492] T 4754eae416d54113a371e6a728514376 P fe7ca79af5664fba8c260aee0614d2d4: No bootstrap required, opened a new log
I20260808 08:29:03.930454 23111 ts_tablet_manager.cc:1397] T 4754eae416d54113a371e6a728514376 P 4f9e7d5cc8ff480e95c5bd2c8aa9bb30: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 08:29:03.930516 23110 ts_tablet_manager.cc:1397] T 4754eae416d54113a371e6a728514376 P fe7ca79af5664fba8c260aee0614d2d4: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20260808 08:29:03.930663 23112 tablet_bootstrap.cc:654] T 4754eae416d54113a371e6a728514376 P 41f1798b08854d128ff96c564a91e3e0: Neither blocks nor log segments found. Creating new log.
I20260808 08:29:03.931345 23112 log.cc:824] T 4754eae416d54113a371e6a728514376 P 41f1798b08854d128ff96c564a91e3e0: Log is configured to *not* fsync() on all Append() calls
I20260808 08:29:03.933331 23112 tablet_bootstrap.cc:492] T 4754eae416d54113a371e6a728514376 P 41f1798b08854d128ff96c564a91e3e0: No bootstrap required, opened a new log
I20260808 08:29:03.933516 23112 ts_tablet_manager.cc:1397] T 4754eae416d54113a371e6a728514376 P 41f1798b08854d128ff96c564a91e3e0: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260808 08:29:03.934669 23110 raft_consensus.cc:348] T 4754eae416d54113a371e6a728514376 P fe7ca79af5664fba8c260aee0614d2d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9e7d5cc8ff480e95c5bd2c8aa9bb30" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 33551 } } peers { permanent_uuid: "fe7ca79af5664fba8c260aee0614d2d4" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 37169 } } peers { permanent_uuid: "41f1798b08854d128ff96c564a91e3e0" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44729 } }
I20260808 08:29:03.934669 23111 raft_consensus.cc:348] T 4754eae416d54113a371e6a728514376 P 4f9e7d5cc8ff480e95c5bd2c8aa9bb30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9e7d5cc8ff480e95c5bd2c8aa9bb30" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 33551 } } peers { permanent_uuid: "fe7ca79af5664fba8c260aee0614d2d4" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 37169 } } peers { permanent_uuid: "41f1798b08854d128ff96c564a91e3e0" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44729 } }
I20260808 08:29:03.934937 23110 raft_consensus.cc:374] T 4754eae416d54113a371e6a728514376 P fe7ca79af5664fba8c260aee0614d2d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:29:03.934937 23111 raft_consensus.cc:374] T 4754eae416d54113a371e6a728514376 P 4f9e7d5cc8ff480e95c5bd2c8aa9bb30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:29:03.934998 23110 raft_consensus.cc:729] T 4754eae416d54113a371e6a728514376 P fe7ca79af5664fba8c260aee0614d2d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe7ca79af5664fba8c260aee0614d2d4, State: Initialized, Role: FOLLOWER
I20260808 08:29:03.935009 23111 raft_consensus.cc:729] T 4754eae416d54113a371e6a728514376 P 4f9e7d5cc8ff480e95c5bd2c8aa9bb30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f9e7d5cc8ff480e95c5bd2c8aa9bb30, State: Initialized, Role: FOLLOWER
I20260808 08:29:03.935467 23110 consensus_queue.cc:260] T 4754eae416d54113a371e6a728514376 P fe7ca79af5664fba8c260aee0614d2d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9e7d5cc8ff480e95c5bd2c8aa9bb30" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 33551 } } peers { permanent_uuid: "fe7ca79af5664fba8c260aee0614d2d4" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 37169 } } peers { permanent_uuid: "41f1798b08854d128ff96c564a91e3e0" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44729 } }
I20260808 08:29:03.935467 23111 consensus_queue.cc:260] T 4754eae416d54113a371e6a728514376 P 4f9e7d5cc8ff480e95c5bd2c8aa9bb30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9e7d5cc8ff480e95c5bd2c8aa9bb30" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 33551 } } peers { permanent_uuid: "fe7ca79af5664fba8c260aee0614d2d4" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 37169 } } peers { permanent_uuid: "41f1798b08854d128ff96c564a91e3e0" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44729 } }
I20260808 08:29:03.936355 22861 heartbeater.cc:503] Master 127.22.235.254:35389 was elected leader, sending a full tablet report...
I20260808 08:29:03.936470 23110 ts_tablet_manager.cc:1428] T 4754eae416d54113a371e6a728514376 P fe7ca79af5664fba8c260aee0614d2d4: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260808 08:29:03.936470 23111 ts_tablet_manager.cc:1428] T 4754eae416d54113a371e6a728514376 P 4f9e7d5cc8ff480e95c5bd2c8aa9bb30: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 08:29:03.937225 23112 raft_consensus.cc:348] T 4754eae416d54113a371e6a728514376 P 41f1798b08854d128ff96c564a91e3e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9e7d5cc8ff480e95c5bd2c8aa9bb30" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 33551 } } peers { permanent_uuid: "fe7ca79af5664fba8c260aee0614d2d4" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 37169 } } peers { permanent_uuid: "41f1798b08854d128ff96c564a91e3e0" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44729 } }
I20260808 08:29:03.937479 23112 raft_consensus.cc:374] T 4754eae416d54113a371e6a728514376 P 41f1798b08854d128ff96c564a91e3e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:29:03.937552 22639 heartbeater.cc:503] Master 127.22.235.254:35389 was elected leader, sending a full tablet report...
I20260808 08:29:03.937554 23112 raft_consensus.cc:729] T 4754eae416d54113a371e6a728514376 P 41f1798b08854d128ff96c564a91e3e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41f1798b08854d128ff96c564a91e3e0, State: Initialized, Role: FOLLOWER
I20260808 08:29:03.938007 23112 consensus_queue.cc:260] T 4754eae416d54113a371e6a728514376 P 41f1798b08854d128ff96c564a91e3e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9e7d5cc8ff480e95c5bd2c8aa9bb30" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 33551 } } peers { permanent_uuid: "fe7ca79af5664fba8c260aee0614d2d4" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 37169 } } peers { permanent_uuid: "41f1798b08854d128ff96c564a91e3e0" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44729 } }
I20260808 08:29:03.939428 23112 ts_tablet_manager.cc:1428] T 4754eae416d54113a371e6a728514376 P 41f1798b08854d128ff96c564a91e3e0: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 08:29:03.939749 23081 heartbeater.cc:503] Master 127.22.235.254:35389 was elected leader, sending a full tablet report...
W20260808 08:29:03.963691 22640 tablet.cc:2369] T 4754eae416d54113a371e6a728514376 P fe7ca79af5664fba8c260aee0614d2d4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 08:29:04.116495 23082 tablet.cc:2369] T 4754eae416d54113a371e6a728514376 P 41f1798b08854d128ff96c564a91e3e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 08:29:04.155766 22862 tablet.cc:2369] T 4754eae416d54113a371e6a728514376 P 4f9e7d5cc8ff480e95c5bd2c8aa9bb30: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 08:29:04.174254 23117 raft_consensus.cc:482] T 4754eae416d54113a371e6a728514376 P fe7ca79af5664fba8c260aee0614d2d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 08:29:04.174485 23117 raft_consensus.cc:504] T 4754eae416d54113a371e6a728514376 P fe7ca79af5664fba8c260aee0614d2d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9e7d5cc8ff480e95c5bd2c8aa9bb30" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 33551 } } peers { permanent_uuid: "fe7ca79af5664fba8c260aee0614d2d4" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 37169 } } peers { permanent_uuid: "41f1798b08854d128ff96c564a91e3e0" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44729 } }
I20260808 08:29:04.175688 23117 leader_election.cc:288] T 4754eae416d54113a371e6a728514376 P fe7ca79af5664fba8c260aee0614d2d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f9e7d5cc8ff480e95c5bd2c8aa9bb30 (127.22.235.194:33551), 41f1798b08854d128ff96c564a91e3e0 (127.22.235.195:44729)
I20260808 08:29:04.180999 23106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:04.176039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:57985 (local address 127.22.235.194:33551)
0808 08:29:04.176297 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:04.176300 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:04.176380 (+ 80us) server_negotiation.cc:407] Connection header received
0808 08:29:04.176636 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:04.176640 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:04.176690 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:04.176776 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:29:04.177153 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:04.177962 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:04.179658 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:04.180255 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:04.180416 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:04.180458 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:04.180796 (+ 338us) server_negotiation.cc:299] Negotiation successful
0808 08:29:04.180864 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 08:29:04.180998 23109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:04.176095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:48759 (local address 127.22.235.195:44729)
0808 08:29:04.176228 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:04.176233 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:04.176550 (+ 317us) server_negotiation.cc:407] Connection header received
0808 08:29:04.176610 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:04.176637 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:04.176690 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:04.176776 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:29:04.177153 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:04.177962 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:04.179658 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:04.180288 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:04.180394 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:04.180448 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:04.180796 (+ 348us) server_negotiation.cc:299] Negotiation successful
0808 08:29:04.180867 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 08:29:04.181128 23121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:04.175950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.195:44729 (local address 127.22.235.193:48759)
0808 08:29:04.176395 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:04.176408 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:04.176485 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:04.176789 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:04.176792 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:04.176835 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:29:04.177030 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:04.177037 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:04.178064 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:04.178067 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:04.179540 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:04.179546 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:04.180423 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:04.180428 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:04.180636 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:04.180696 (+ 60us) client_negotiation.cc:769] Sending connection context
0808 08:29:04.180781 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 08:29:04.180913 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"mutex_wait_us":217,"spinlock_wait_cycles":33792,"thread_start_us":108,"threads_started":1}
I20260808 08:29:04.181126 23122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:04.175930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.194:33551 (local address 127.22.235.193:57985)
0808 08:29:04.176283 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:04.176299 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:04.176438 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:04.176789 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:04.176792 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:04.176837 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:29:04.177023 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:04.177031 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:04.178064 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:04.178067 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:04.179533 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:04.179546 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:04.180423 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:04.180428 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:04.180640 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:04.180696 (+ 56us) client_negotiation.cc:769] Sending connection context
0808 08:29:04.180781 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 08:29:04.180913 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":214,"spinlock_wait_cycles":4992,"thread_start_us":88,"threads_started":1}
I20260808 08:29:04.181811 22783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4754eae416d54113a371e6a728514376" candidate_uuid: "fe7ca79af5664fba8c260aee0614d2d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f9e7d5cc8ff480e95c5bd2c8aa9bb30" is_pre_election: true
I20260808 08:29:04.181939 22993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4754eae416d54113a371e6a728514376" candidate_uuid: "fe7ca79af5664fba8c260aee0614d2d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41f1798b08854d128ff96c564a91e3e0" is_pre_election: true
I20260808 08:29:04.182154 22783 raft_consensus.cc:2454] T 4754eae416d54113a371e6a728514376 P 4f9e7d5cc8ff480e95c5bd2c8aa9bb30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe7ca79af5664fba8c260aee0614d2d4 in term 0.
I20260808 08:29:04.182233 22993 raft_consensus.cc:2454] T 4754eae416d54113a371e6a728514376 P 41f1798b08854d128ff96c564a91e3e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe7ca79af5664fba8c260aee0614d2d4 in term 0.
I20260808 08:29:04.182638 22460 leader_election.cc:302] T 4754eae416d54113a371e6a728514376 P fe7ca79af5664fba8c260aee0614d2d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 41f1798b08854d128ff96c564a91e3e0, 4f9e7d5cc8ff480e95c5bd2c8aa9bb30, fe7ca79af5664fba8c260aee0614d2d4; no voters:
I20260808 08:29:04.182929 23117 raft_consensus.cc:2781] T 4754eae416d54113a371e6a728514376 P fe7ca79af5664fba8c260aee0614d2d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 08:29:04.183033 23117 raft_consensus.cc:482] T 4754eae416d54113a371e6a728514376 P fe7ca79af5664fba8c260aee0614d2d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 08:29:04.183089 23117 raft_consensus.cc:3037] T 4754eae416d54113a371e6a728514376 P fe7ca79af5664fba8c260aee0614d2d4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:29:04.184180 23117 raft_consensus.cc:504] T 4754eae416d54113a371e6a728514376 P fe7ca79af5664fba8c260aee0614d2d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9e7d5cc8ff480e95c5bd2c8aa9bb30" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 33551 } } peers { permanent_uuid: "fe7ca79af5664fba8c260aee0614d2d4" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 37169 } } peers { permanent_uuid: "41f1798b08854d128ff96c564a91e3e0" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44729 } }
I20260808 08:29:04.184510 23117 leader_election.cc:288] T 4754eae416d54113a371e6a728514376 P fe7ca79af5664fba8c260aee0614d2d4 [CANDIDATE]: Term 1 election: Requested vote from peers 4f9e7d5cc8ff480e95c5bd2c8aa9bb30 (127.22.235.194:33551), 41f1798b08854d128ff96c564a91e3e0 (127.22.235.195:44729)
I20260808 08:29:04.184871 22783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4754eae416d54113a371e6a728514376" candidate_uuid: "fe7ca79af5664fba8c260aee0614d2d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f9e7d5cc8ff480e95c5bd2c8aa9bb30"
I20260808 08:29:04.184883 22993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4754eae416d54113a371e6a728514376" candidate_uuid: "fe7ca79af5664fba8c260aee0614d2d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41f1798b08854d128ff96c564a91e3e0"
I20260808 08:29:04.184998 22993 raft_consensus.cc:3037] T 4754eae416d54113a371e6a728514376 P 41f1798b08854d128ff96c564a91e3e0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:29:04.185022 22783 raft_consensus.cc:3037] T 4754eae416d54113a371e6a728514376 P 4f9e7d5cc8ff480e95c5bd2c8aa9bb30 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:29:04.186239 22993 raft_consensus.cc:2454] T 4754eae416d54113a371e6a728514376 P 41f1798b08854d128ff96c564a91e3e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe7ca79af5664fba8c260aee0614d2d4 in term 1.
I20260808 08:29:04.186239 22783 raft_consensus.cc:2454] T 4754eae416d54113a371e6a728514376 P 4f9e7d5cc8ff480e95c5bd2c8aa9bb30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe7ca79af5664fba8c260aee0614d2d4 in term 1.
I20260808 08:29:04.186563 22461 leader_election.cc:302] T 4754eae416d54113a371e6a728514376 P fe7ca79af5664fba8c260aee0614d2d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41f1798b08854d128ff96c564a91e3e0, fe7ca79af5664fba8c260aee0614d2d4; no voters:
I20260808 08:29:04.186740 23117 raft_consensus.cc:2781] T 4754eae416d54113a371e6a728514376 P fe7ca79af5664fba8c260aee0614d2d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:29:04.186992 23117 raft_consensus.cc:686] T 4754eae416d54113a371e6a728514376 P fe7ca79af5664fba8c260aee0614d2d4 [term 1 LEADER]: Becoming Leader. State: Replica: fe7ca79af5664fba8c260aee0614d2d4, State: Running, Role: LEADER
I20260808 08:29:04.187363 23117 consensus_queue.cc:237] T 4754eae416d54113a371e6a728514376 P fe7ca79af5664fba8c260aee0614d2d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9e7d5cc8ff480e95c5bd2c8aa9bb30" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 33551 } } peers { permanent_uuid: "fe7ca79af5664fba8c260aee0614d2d4" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 37169 } } peers { permanent_uuid: "41f1798b08854d128ff96c564a91e3e0" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44729 } }
I20260808 08:29:04.190451 22324 catalog_manager.cc:5495] T 4754eae416d54113a371e6a728514376 P fe7ca79af5664fba8c260aee0614d2d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to fe7ca79af5664fba8c260aee0614d2d4 (127.22.235.193). New cstate: current_term: 1 leader_uuid: "fe7ca79af5664fba8c260aee0614d2d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f9e7d5cc8ff480e95c5bd2c8aa9bb30" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 33551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fe7ca79af5664fba8c260aee0614d2d4" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 37169 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41f1798b08854d128ff96c564a91e3e0" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44729 } health_report { overall_health: UNKNOWN } } }
I20260808 08:29:04.206763 23107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:04.203236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37574 (local address 127.22.235.193:37169)
0808 08:29:04.203345 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:04.203349 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:04.203363 (+ 14us) server_negotiation.cc:407] Connection header received
0808 08:29:04.203535 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:04.203537 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:04.203590 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:04.203674 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:29:04.203908 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:04.204388 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:04.205370 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:04.205587 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:04.205672 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:04.205965 (+ 293us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:29:04.206350 (+ 385us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 08:29:04.206477 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:04.206592 (+ 115us) server_negotiation.cc:299] Negotiation successful
0808 08:29:04.206626 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 08:29:04.212399 22783 raft_consensus.cc:1261] T 4754eae416d54113a371e6a728514376 P 4f9e7d5cc8ff480e95c5bd2c8aa9bb30 [term 1 FOLLOWER]: Refusing update from remote peer fe7ca79af5664fba8c260aee0614d2d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:29:04.212399 22993 raft_consensus.cc:1261] T 4754eae416d54113a371e6a728514376 P 41f1798b08854d128ff96c564a91e3e0 [term 1 FOLLOWER]: Refusing update from remote peer fe7ca79af5664fba8c260aee0614d2d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:29:04.213119 23117 consensus_queue.cc:1035] T 4754eae416d54113a371e6a728514376 P fe7ca79af5664fba8c260aee0614d2d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f9e7d5cc8ff480e95c5bd2c8aa9bb30" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 33551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:29:04.213317 23123 consensus_queue.cc:1035] T 4754eae416d54113a371e6a728514376 P fe7ca79af5664fba8c260aee0614d2d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41f1798b08854d128ff96c564a91e3e0" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 44729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:29:04.219554 23127 mvcc.cc:205] Tried to move back new op lower bound from 7316184040286494720 to 7316184040193732608. Current Snapshot: MvccSnapshot[applied={T|T < 7316184040286494720}]
I20260808 08:29:04.224836 23129 mvcc.cc:205] Tried to move back new op lower bound from 7316184040286494720 to 7316184040193732608. Current Snapshot: MvccSnapshot[applied={T|T < 7316184040286494720}]
I20260808 08:29:04.229607 22324 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:29:04.229768 22324 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:29:04.232246 22324 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 87fac91252d2486f89e9edbb8cfcac45: Sending DeleteTablet for 3 replicas of tablet 4754eae416d54113a371e6a728514376
I20260808 08:29:04.233049 22752 tablet_service.cc:1514] Processing DeleteTablet for tablet 4754eae416d54113a371e6a728514376 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:29:04 UTC) from {username='slave'} at 127.0.0.1:42250
I20260808 08:29:04.233054 22533 tablet_service.cc:1514] Processing DeleteTablet for tablet 4754eae416d54113a371e6a728514376 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:29:04 UTC) from {username='slave'} at 127.0.0.1:37568
I20260808 08:29:04.233083 22976 tablet_service.cc:1514] Processing DeleteTablet for tablet 4754eae416d54113a371e6a728514376 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:29:04 UTC) from {username='slave'} at 127.0.0.1:59010
I20260808 08:29:04.233897 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 22434
I20260808 08:29:04.244001 23143 tablet_replica.cc:330] T 4754eae416d54113a371e6a728514376 P 4f9e7d5cc8ff480e95c5bd2c8aa9bb30: stopping tablet replica
I20260808 08:29:04.244387 23143 raft_consensus.cc:2229] T 4754eae416d54113a371e6a728514376 P 4f9e7d5cc8ff480e95c5bd2c8aa9bb30 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 08:29:04.244784 23143 raft_consensus.cc:2258] T 4754eae416d54113a371e6a728514376 P 4f9e7d5cc8ff480e95c5bd2c8aa9bb30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:29:04.247325 23143 ts_tablet_manager.cc:1905] T 4754eae416d54113a371e6a728514376 P 4f9e7d5cc8ff480e95c5bd2c8aa9bb30: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:29:04.248885 23142 tablet_replica.cc:330] T 4754eae416d54113a371e6a728514376 P 41f1798b08854d128ff96c564a91e3e0: stopping tablet replica
I20260808 08:29:04.249226 23142 raft_consensus.cc:2229] T 4754eae416d54113a371e6a728514376 P 41f1798b08854d128ff96c564a91e3e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 08:29:04.249568 23142 raft_consensus.cc:2258] T 4754eae416d54113a371e6a728514376 P 41f1798b08854d128ff96c564a91e3e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:29:04.250270 23143 ts_tablet_manager.cc:1918] T 4754eae416d54113a371e6a728514376 P 4f9e7d5cc8ff480e95c5bd2c8aa9bb30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 08:29:04.250365 23143 log.cc:1196] T 4754eae416d54113a371e6a728514376 P 4f9e7d5cc8ff480e95c5bd2c8aa9bb30: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-1/wal/wals/4754eae416d54113a371e6a728514376
I20260808 08:29:04.250638 23143 ts_tablet_manager.cc:1939] T 4754eae416d54113a371e6a728514376 P 4f9e7d5cc8ff480e95c5bd2c8aa9bb30: Deleting consensus metadata
I20260808 08:29:04.251185 23142 ts_tablet_manager.cc:1905] T 4754eae416d54113a371e6a728514376 P 41f1798b08854d128ff96c564a91e3e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:29:04.251793 22306 catalog_manager.cc:4841] TS 4f9e7d5cc8ff480e95c5bd2c8aa9bb30 (127.22.235.194:33551): tablet 4754eae416d54113a371e6a728514376 (table test-table [id=133bbbc523d24c559d057e31543e30e1]) successfully deleted
I20260808 08:29:04.253793 23142 ts_tablet_manager.cc:1918] T 4754eae416d54113a371e6a728514376 P 41f1798b08854d128ff96c564a91e3e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 08:29:04.253890 23142 log.cc:1196] T 4754eae416d54113a371e6a728514376 P 41f1798b08854d128ff96c564a91e3e0: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786177493204855-23471-0/minicluster-data/ts-2/wal/wals/4754eae416d54113a371e6a728514376
I20260808 08:29:04.254151 23142 ts_tablet_manager.cc:1939] T 4754eae416d54113a371e6a728514376 P 41f1798b08854d128ff96c564a91e3e0: Deleting consensus metadata
I20260808 08:29:04.255275 22307 catalog_manager.cc:4841] TS 41f1798b08854d128ff96c564a91e3e0 (127.22.235.195:44729): tablet 4754eae416d54113a371e6a728514376 (table test-table [id=133bbbc523d24c559d057e31543e30e1]) successfully deleted
W20260808 08:29:04.256276 22304 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.235.193:37169 (error 108)
I20260808 08:29:04.256558 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 22645
I20260808 08:29:04.266635 23108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:04.266223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:37169 (local address 127.0.0.1:37588)
0808 08:29:04.266413 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:04.266526 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.193:37169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260808 08:29:04.266860 22304 catalog_manager.cc:4584] TS fe7ca79af5664fba8c260aee0614d2d4 (127.22.235.193:37169): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4754eae416d54113a371e6a728514376: Network error: Client connection negotiation failed: client connection to 127.22.235.193:37169: connect: Connection refused (error 111)
I20260808 08:29:04.271919 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 22866
I20260808 08:29:04.280864 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 22283
2026-08-08T08:29:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1271 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 08:29:04 dist-test-slave-f7th krb5kdc[23149](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 08:29:04 dist-test-slave-f7th krb5kdc[23149](info): set up 2 sockets
Aug 08 08:29:04 dist-test-slave-f7th krb5kdc[23149](info): commencing operation
krb5kdc: starting...
W20260808 08:29:05.444557 23471 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Aug 08 08:29:05 dist-test-slave-f7th krb5kdc[23149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177745, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T08:29:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:29:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.235.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
I20260808 08:29:05.561369 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:35779
--webserver_interface=127.22.235.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:40419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:35779
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:29:05.673143 23173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:29:05.673398 23173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:29:05.673455 23173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:29:05.677366 23173 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:29:05.677418 23173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:29:05.677445 23173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:29:05.677469 23173 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:29:05.677497 23173 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:29:05.683058 23173 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:40419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:35779
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:35779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.235.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:29:05.684052 23173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:29:05.685075 23173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:29:05.693324 23183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:05.693413 23192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:05.693346 23185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:05.695641 23173 server_base.cc:1034] running on GCE node
I20260808 08:29:05.696188 23173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:29:05.697342 23173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:29:05.698501 23173 hybrid_clock.cc:648] HybridClock initialized: now 1786177745698472 us; error 39 us; skew 500 ppm
Aug 08 08:29:05 dist-test-slave-f7th krb5kdc[23149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177745, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:29:05.701241 23173 init.cc:377] Logged in from keytab as kudu/127.22.235.254@KRBTEST.COM (short username kudu)
I20260808 08:29:05.702157 23173 webserver.cc:460] Webserver started at http://127.22.235.254:39185/ using document root <none> and password file <none>
I20260808 08:29:05.702709 23173 fs_manager.cc:359] Metadata directory not provided
I20260808 08:29:05.702780 23173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:29:05.702972 23173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:29:05.705053 23173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "2985a84ac7d242e6806ad1c4ab0452c4"
format_stamp: "Formatted at 2026-08-08 08:29:05 on dist-test-slave-f7th"
server_key: "5ceb0c088a02d6cb822bca97ca371561"
server_key_iv: "e1b0255d5a047c8117b8326997b6e495"
server_key_version: "encryptionkey@0"
I20260808 08:29:05.705586 23173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "2985a84ac7d242e6806ad1c4ab0452c4"
format_stamp: "Formatted at 2026-08-08 08:29:05 on dist-test-slave-f7th"
server_key: "5ceb0c088a02d6cb822bca97ca371561"
server_key_iv: "e1b0255d5a047c8117b8326997b6e495"
server_key_version: "encryptionkey@0"
I20260808 08:29:05.709218 23173 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 08:29:05.711486 23204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:05.712771 23173 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:29:05.712898 23173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "2985a84ac7d242e6806ad1c4ab0452c4"
format_stamp: "Formatted at 2026-08-08 08:29:05 on dist-test-slave-f7th"
server_key: "5ceb0c088a02d6cb822bca97ca371561"
server_key_iv: "e1b0255d5a047c8117b8326997b6e495"
server_key_version: "encryptionkey@0"
I20260808 08:29:05.713008 23173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:29:05.732014 23173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:29:05.734704 23173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:29:05.734877 23173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:29:05.741452 23173 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.254:35779
I20260808 08:29:05.741464 23279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.254:35779 every 8 connection(s)
I20260808 08:29:05.742372 23173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:29:05.745150 23281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:29:05.747056 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 23173
I20260808 08:29:05.747182 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:29:05.747430 23471 external_mini_cluster.cc:1428] Setting key 76c12622a028fce1a801e0bde01d3f4b
I20260808 08:29:05.751886 23281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2985a84ac7d242e6806ad1c4ab0452c4: Bootstrap starting.
I20260808 08:29:05.754519 23281 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2985a84ac7d242e6806ad1c4ab0452c4: Neither blocks nor log segments found. Creating new log.
I20260808 08:29:05.755342 23281 log.cc:824] T 00000000000000000000000000000000 P 2985a84ac7d242e6806ad1c4ab0452c4: Log is configured to *not* fsync() on all Append() calls
Aug 08 08:29:05 dist-test-slave-f7th krb5kdc[23149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177745, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:29:05.757521 23281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2985a84ac7d242e6806ad1c4ab0452c4: No bootstrap required, opened a new log
I20260808 08:29:05.762003 23281 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2985a84ac7d242e6806ad1c4ab0452c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2985a84ac7d242e6806ad1c4ab0452c4" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 35779 } }
I20260808 08:29:05.762221 23285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:05.748783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59534 (local address 127.22.235.254:35779)
0808 08:29:05.749206 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:05.749225 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:05.749268 (+ 43us) server_negotiation.cc:407] Connection header received
0808 08:29:05.749994 (+ 726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:05.750019 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:05.750430 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:05.750795 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:05.751421 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:05.752889 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:05.753735 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:05.754397 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:05.754566 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:05.757194 (+ 2628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:05.757202 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:05.757218 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:05.757250 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:05.759111 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:05.759544 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:05.759550 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:05.759556 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:05.759708 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:05.759941 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:05.759944 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:05.759947 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:05.760445 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:05.760697 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:05.760948 (+ 251us) server_negotiation.cc:299] Negotiation successful
0808 08:29:05.761256 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":108,"threads_started":1}
I20260808 08:29:05.762272 23281 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2985a84ac7d242e6806ad1c4ab0452c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:29:05.762317 23281 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2985a84ac7d242e6806ad1c4ab0452c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2985a84ac7d242e6806ad1c4ab0452c4, State: Initialized, Role: FOLLOWER
I20260808 08:29:05.762758 23281 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2985a84ac7d242e6806ad1c4ab0452c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2985a84ac7d242e6806ad1c4ab0452c4" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 35779 } }
I20260808 08:29:05.762895 23281 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2985a84ac7d242e6806ad1c4ab0452c4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:29:05.762959 23281 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2985a84ac7d242e6806ad1c4ab0452c4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:29:05.763063 23281 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2985a84ac7d242e6806ad1c4ab0452c4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:29:05.763916 23281 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2985a84ac7d242e6806ad1c4ab0452c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2985a84ac7d242e6806ad1c4ab0452c4" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 35779 } }
I20260808 08:29:05.764245 23281 leader_election.cc:302] T 00000000000000000000000000000000 P 2985a84ac7d242e6806ad1c4ab0452c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2985a84ac7d242e6806ad1c4ab0452c4; no voters:
I20260808 08:29:05.764550 23281 leader_election.cc:288] T 00000000000000000000000000000000 P 2985a84ac7d242e6806ad1c4ab0452c4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:29:05.764644 23287 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2985a84ac7d242e6806ad1c4ab0452c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:29:05.764842 23287 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2985a84ac7d242e6806ad1c4ab0452c4 [term 1 LEADER]: Becoming Leader. State: Replica: 2985a84ac7d242e6806ad1c4ab0452c4, State: Running, Role: LEADER
I20260808 08:29:05.765173 23287 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2985a84ac7d242e6806ad1c4ab0452c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2985a84ac7d242e6806ad1c4ab0452c4" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 35779 } }
I20260808 08:29:05.765877 23281 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2985a84ac7d242e6806ad1c4ab0452c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:29:05.766991 23288 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2985a84ac7d242e6806ad1c4ab0452c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2985a84ac7d242e6806ad1c4ab0452c4. Latest consensus state: current_term: 1 leader_uuid: "2985a84ac7d242e6806ad1c4ab0452c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2985a84ac7d242e6806ad1c4ab0452c4" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 35779 } } }
I20260808 08:29:05.767138 23288 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2985a84ac7d242e6806ad1c4ab0452c4 [sys.catalog]: This master's current role is: LEADER
I20260808 08:29:05.767005 23290 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2985a84ac7d242e6806ad1c4ab0452c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2985a84ac7d242e6806ad1c4ab0452c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2985a84ac7d242e6806ad1c4ab0452c4" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 35779 } } }
I20260808 08:29:05.767473 23290 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2985a84ac7d242e6806ad1c4ab0452c4 [sys.catalog]: This master's current role is: LEADER
I20260808 08:29:05.767583 23302 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:29:05.770741 23302 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:29:05.776099 23302 catalog_manager.cc:1347] Generated new cluster ID: d3f0c18daea04b5087fb135eae56a374
I20260808 08:29:05.776170 23302 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:29:05.802855 23302 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:29:05.804229 23302 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:29:05.813882 23302 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2985a84ac7d242e6806ad1c4ab0452c4: Generated new TSK 0
I20260808 08:29:05.814705 23302 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.235.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab.
Entry for principal kudu/127.22.235.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab.
I20260808 08:29:05.853623 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.193:0
--local_ip_for_outbound_sockets=127.22.235.193
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:35779
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab
--principal=kudu/127.22.235.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:40419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:29:05.988528 23317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:29:05.988807 23317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:29:05.988849 23317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:29:05.992753 23317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:29:05.992807 23317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:29:05.992862 23317 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.193
I20260808 08:29:05.997895 23317 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:40419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.193.keytab
--principal=kudu/127.22.235.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:35779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.193
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:29:05.998836 23317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:29:05.999763 23317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:29:06.009023 23326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:06.008998 23324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:06.009008 23329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:06.009202 23317 server_base.cc:1034] running on GCE node
I20260808 08:29:06.009624 23317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:29:06.010226 23317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:29:06.011386 23317 hybrid_clock.cc:648] HybridClock initialized: now 1786177746011368 us; error 27 us; skew 500 ppm
Aug 08 08:29:06 dist-test-slave-f7th krb5kdc[23149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177746, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:29:06.014056 23317 init.cc:377] Logged in from keytab as kudu/127.22.235.193@KRBTEST.COM (short username kudu)
I20260808 08:29:06.014853 23317 webserver.cc:460] Webserver started at http://127.22.235.193:37543/ using document root <none> and password file <none>
I20260808 08:29:06.015385 23317 fs_manager.cc:359] Metadata directory not provided
I20260808 08:29:06.015450 23317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:29:06.015688 23317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:29:06.017735 23317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-0/data/instance:
uuid: "79d414e06a8348ee87042b72995ed5eb"
format_stamp: "Formatted at 2026-08-08 08:29:06 on dist-test-slave-f7th"
server_key: "b169634ea1307ebd9638c03ea010a2a1"
server_key_iv: "6e4016258c244108257c0688005ec24b"
server_key_version: "encryptionkey@0"
I20260808 08:29:06.018260 23317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance:
uuid: "79d414e06a8348ee87042b72995ed5eb"
format_stamp: "Formatted at 2026-08-08 08:29:06 on dist-test-slave-f7th"
server_key: "b169634ea1307ebd9638c03ea010a2a1"
server_key_iv: "6e4016258c244108257c0688005ec24b"
server_key_version: "encryptionkey@0"
I20260808 08:29:06.021863 23317 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 08:29:06.024214 23337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:06.025380 23317 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:29:06.025483 23317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-0/wal
uuid: "79d414e06a8348ee87042b72995ed5eb"
format_stamp: "Formatted at 2026-08-08 08:29:06 on dist-test-slave-f7th"
server_key: "b169634ea1307ebd9638c03ea010a2a1"
server_key_iv: "6e4016258c244108257c0688005ec24b"
server_key_version: "encryptionkey@0"
I20260808 08:29:06.025576 23317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:29:06.041834 23317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:29:06.044605 23317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:29:06.044788 23317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:29:06.045329 23317 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:29:06.046125 23317 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:29:06.046170 23317 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:06.046213 23317 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:29:06.046234 23317 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:06.054697 23317 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.193:45615
I20260808 08:29:06.054749 23515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.193:45615 every 8 connection(s)
I20260808 08:29:06.055588 23317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
I20260808 08:29:06.060854 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 23317
I20260808 08:29:06.060981 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance
I20260808 08:29:06.061234 23471 external_mini_cluster.cc:1428] Setting key 9b4349648b1a5497bc12ea148a3a888b
Aug 08 08:29:06 dist-test-slave-f7th krb5kdc[23149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177746, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.193@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:29:06.069643 23285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:06.057385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:58121 (local address 127.22.235.254:35779)
0808 08:29:06.057577 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:06.057582 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:06.058301 (+ 719us) server_negotiation.cc:407] Connection header received
0808 08:29:06.059624 (+ 1323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:06.059626 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:06.059673 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:06.059743 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:06.060945 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:06.061552 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:06.062884 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:06.063124 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:06.063204 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:06.065945 (+ 2741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:06.065949 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:06.065951 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:06.065980 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:06.067577 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:06.068048 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:06.068052 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:06.068054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:06.068153 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:06.068456 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:06.068459 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:06.068468 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:06.068731 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:06.068832 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:06.069301 (+ 469us) server_negotiation.cc:299] Negotiation successful
0808 08:29:06.069453 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260808 08:29:06.070410 23520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:06.057714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:35779 (local address 127.22.235.193:58121)
0808 08:29:06.058157 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:06.058197 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:06.059441 (+ 1244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:06.059933 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:06.059940 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:06.060261 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:29:06.060780 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:06.060802 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:06.061683 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:06.061686 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:06.062754 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:06.062767 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:06.063268 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:06.063271 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:06.063453 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:06.064006 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:29:06.064026 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:29:06.065769 (+ 1743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:29:06.067683 (+ 1914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:29:06.067688 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:29:06.067700 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:29:06.067942 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:29:06.068238 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:29:06.068240 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:29:06.068241 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:29:06.068373 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:29:06.068827 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:29:06.068832 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:29:06.069070 (+ 238us) client_negotiation.cc:769] Sending connection context
0808 08:29:06.069224 (+ 154us) client_negotiation.cc:240] Negotiation successful
0808 08:29:06.069474 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":104,"threads_started":1}
I20260808 08:29:06.071601 23516 heartbeater.cc:347] Connected to a master server at 127.22.235.254:35779
I20260808 08:29:06.071874 23516 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.22.235.194@KRBTEST.COM; defaulting to no policy
I20260808 08:29:06.072479 23516 heartbeater.cc:511] Master 127.22.235.254:35779 requested a full tablet report, sending...
I20260808 08:29:06.073979 23230 ts_manager.cc:194] Registered new tserver with Master: 79d414e06a8348ee87042b72995ed5eb (127.22.235.193:45615)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.194@KRBTEST.COM" created.
I20260808 08:29:06.075184 23230 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.193@KRBTEST.COM'} at 127.22.235.193:58121
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab.
Entry for principal kudu/127.22.235.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab.
I20260808 08:29:06.089464 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.194:0
--local_ip_for_outbound_sockets=127.22.235.194
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:35779
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab
--principal=kudu/127.22.235.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:40419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:29:06.199193 23523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:29:06.199445 23523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:29:06.199499 23523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:29:06.203583 23523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:29:06.203644 23523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:29:06.203722 23523 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.194
I20260808 08:29:06.208882 23523 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:40419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.194.keytab
--principal=kudu/127.22.235.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:35779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23523
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.194
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:29:06.209914 23523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:29:06.210925 23523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 08:29:06.220398 23523 server_base.cc:1034] running on GCE node
W20260808 08:29:06.220381 23534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:06.220381 23536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:06.220372 23539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:06.221094 23523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:29:06.221719 23523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:29:06.222874 23523 hybrid_clock.cc:648] HybridClock initialized: now 1786177746222859 us; error 27 us; skew 500 ppm
Aug 08 08:29:06 dist-test-slave-f7th krb5kdc[23149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177746, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:29:06.225503 23523 init.cc:377] Logged in from keytab as kudu/127.22.235.194@KRBTEST.COM (short username kudu)
I20260808 08:29:06.226302 23523 webserver.cc:460] Webserver started at http://127.22.235.194:44895/ using document root <none> and password file <none>
I20260808 08:29:06.227005 23523 fs_manager.cc:359] Metadata directory not provided
I20260808 08:29:06.227088 23523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:29:06.227320 23523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:29:06.229503 23523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-1/data/instance:
uuid: "92ba0a4bb75244baa0df2c1e6ad990c8"
format_stamp: "Formatted at 2026-08-08 08:29:06 on dist-test-slave-f7th"
server_key: "84eee07703f8de7e35b5fd0167d29461"
server_key_iv: "2c637b9e1b97250e30ffc6417e5cc195"
server_key_version: "encryptionkey@0"
I20260808 08:29:06.230036 23523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance:
uuid: "92ba0a4bb75244baa0df2c1e6ad990c8"
format_stamp: "Formatted at 2026-08-08 08:29:06 on dist-test-slave-f7th"
server_key: "84eee07703f8de7e35b5fd0167d29461"
server_key_iv: "2c637b9e1b97250e30ffc6417e5cc195"
server_key_version: "encryptionkey@0"
I20260808 08:29:06.233652 23523 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 08:29:06.235944 23553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:06.237156 23523 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:29:06.237262 23523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-1/wal
uuid: "92ba0a4bb75244baa0df2c1e6ad990c8"
format_stamp: "Formatted at 2026-08-08 08:29:06 on dist-test-slave-f7th"
server_key: "84eee07703f8de7e35b5fd0167d29461"
server_key_iv: "2c637b9e1b97250e30ffc6417e5cc195"
server_key_version: "encryptionkey@0"
I20260808 08:29:06.237350 23523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:29:06.270569 23523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:29:06.273352 23523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:29:06.273536 23523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:29:06.274103 23523 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:29:06.274930 23523 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:29:06.274977 23523 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:06.275031 23523 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:29:06.275058 23523 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:06.283408 23523 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.194:40613
I20260808 08:29:06.283438 23726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.194:40613 every 8 connection(s)
I20260808 08:29:06.284276 23523 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
I20260808 08:29:06.285415 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 23523
I20260808 08:29:06.285526 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance
I20260808 08:29:06.285820 23471 external_mini_cluster.cc:1428] Setting key aec4ca5d29d2f4541f9fd72b4df8be4b
Aug 08 08:29:06 dist-test-slave-f7th krb5kdc[23149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177746, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.194@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.235.195@KRBTEST.COM; defaulting to no policy
I20260808 08:29:06.298156 23285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:06.286050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:38779 (local address 127.22.235.254:35779)
0808 08:29:06.286168 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:06.286172 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:06.287160 (+ 988us) server_negotiation.cc:407] Connection header received
0808 08:29:06.288517 (+ 1357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:06.288520 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:06.288573 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:06.288690 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:06.289768 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:06.290316 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:06.291403 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:06.291705 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:06.291797 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:06.294384 (+ 2587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:06.294388 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:06.294391 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:06.294423 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:06.295897 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:06.296513 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:06.296518 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:06.296520 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:06.296684 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:06.297070 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:06.297072 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:06.297073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:06.297250 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:06.297326 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:06.297915 (+ 589us) server_negotiation.cc:299] Negotiation successful
0808 08:29:06.298028 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 08:29:06.298895 23730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:06.286509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:35779 (local address 127.22.235.194:38779)
0808 08:29:06.287016 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:06.287067 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:06.288276 (+ 1209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:06.288822 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:06.288828 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:06.289170 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:29:06.289629 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:06.289643 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:06.290432 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:06.290435 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:06.291286 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:06.291297 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:06.291861 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:06.291864 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:06.292021 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:06.292556 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:29:06.292572 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:29:06.294215 (+ 1643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:29:06.296033 (+ 1818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:29:06.296039 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:29:06.296055 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:29:06.296380 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:29:06.296798 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:29:06.296801 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:29:06.296803 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:29:06.296981 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:29:06.297341 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:29:06.297348 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:29:06.297663 (+ 315us) client_negotiation.cc:769] Sending connection context
0808 08:29:06.297850 (+ 187us) client_negotiation.cc:240] Negotiation successful
0808 08:29:06.298094 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":106,"threads_started":1}
I20260808 08:29:06.299947 23728 heartbeater.cc:347] Connected to a master server at 127.22.235.254:35779
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.195@KRBTEST.COM" created.
I20260808 08:29:06.300182 23728 heartbeater.cc:464] Registering TS with master...
I20260808 08:29:06.300758 23728 heartbeater.cc:511] Master 127.22.235.254:35779 requested a full tablet report, sending...
I20260808 08:29:06.301887 23230 ts_manager.cc:194] Registered new tserver with Master: 92ba0a4bb75244baa0df2c1e6ad990c8 (127.22.235.194:40613)
I20260808 08:29:06.302438 23230 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.194@KRBTEST.COM'} at 127.22.235.194:38779
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab.
Entry for principal kudu/127.22.235.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab.
I20260808 08:29:06.315044 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.195:0
--local_ip_for_outbound_sockets=127.22.235.195
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:35779
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab
--principal=kudu/127.22.235.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:40419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:29:06.426146 23733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:29:06.426396 23733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:29:06.426455 23733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:29:06.430460 23733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:29:06.430513 23733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:29:06.430589 23733 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.195
I20260808 08:29:06.435704 23733 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:40419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.195.keytab
--principal=kudu/127.22.235.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:35779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.195
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:29:06.436746 23733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:29:06.437737 23733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:29:06.447294 23744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:06.447311 23740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:06.447299 23741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:06.447615 23733 server_base.cc:1034] running on GCE node
I20260808 08:29:06.448071 23733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:29:06.448714 23733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:29:06.449869 23733 hybrid_clock.cc:648] HybridClock initialized: now 1786177746449853 us; error 31 us; skew 500 ppm
Aug 08 08:29:06 dist-test-slave-f7th krb5kdc[23149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177746, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:29:06.452389 23733 init.cc:377] Logged in from keytab as kudu/127.22.235.195@KRBTEST.COM (short username kudu)
I20260808 08:29:06.453258 23733 webserver.cc:460] Webserver started at http://127.22.235.195:40221/ using document root <none> and password file <none>
I20260808 08:29:06.453779 23733 fs_manager.cc:359] Metadata directory not provided
I20260808 08:29:06.453845 23733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:29:06.454032 23733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:29:06.455967 23733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-2/data/instance:
uuid: "88f23151d47a472681795eba527ee6f9"
format_stamp: "Formatted at 2026-08-08 08:29:06 on dist-test-slave-f7th"
server_key: "8d35f7ca5cfc0486ff1a226d20427f21"
server_key_iv: "e2fdbe132e7944b188e7b4fe0ce110ea"
server_key_version: "encryptionkey@0"
I20260808 08:29:06.456533 23733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance:
uuid: "88f23151d47a472681795eba527ee6f9"
format_stamp: "Formatted at 2026-08-08 08:29:06 on dist-test-slave-f7th"
server_key: "8d35f7ca5cfc0486ff1a226d20427f21"
server_key_iv: "e2fdbe132e7944b188e7b4fe0ce110ea"
server_key_version: "encryptionkey@0"
I20260808 08:29:06.460062 23733 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 08:29:06.462427 23754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:06.463579 23733 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:29:06.463683 23733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-2/wal
uuid: "88f23151d47a472681795eba527ee6f9"
format_stamp: "Formatted at 2026-08-08 08:29:06 on dist-test-slave-f7th"
server_key: "8d35f7ca5cfc0486ff1a226d20427f21"
server_key_iv: "e2fdbe132e7944b188e7b4fe0ce110ea"
server_key_version: "encryptionkey@0"
I20260808 08:29:06.463771 23733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:29:06.492247 23733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:29:06.495070 23733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:29:06.495244 23733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:29:06.495831 23733 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:29:06.496728 23733 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:29:06.496779 23733 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:06.496836 23733 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:29:06.496860 23733 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:06.505328 23733 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.195:39219
I20260808 08:29:06.505357 23924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.195:39219 every 8 connection(s)
I20260808 08:29:06.506182 23733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
I20260808 08:29:06.511094 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 23733
I20260808 08:29:06.511219 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance
I20260808 08:29:06.511472 23471 external_mini_cluster.cc:1428] Setting key a71fdde076d62eacd53008470a68550b
Aug 08 08:29:06 dist-test-slave-f7th krb5kdc[23149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177746, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.195@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:29:06.519675 23285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:06.507943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:47527 (local address 127.22.235.254:35779)
0808 08:29:06.508075 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:06.508079 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:06.508933 (+ 854us) server_negotiation.cc:407] Connection header received
0808 08:29:06.510236 (+ 1303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:06.510239 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:06.510299 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:06.510372 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:06.511616 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:06.512216 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:06.513513 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:06.513804 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:06.513911 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:06.516253 (+ 2342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:06.516256 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:06.516258 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:06.516282 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:06.517710 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:06.518179 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:06.518182 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:06.518183 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:06.518264 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:06.518573 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:06.518576 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:06.518578 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:06.518755 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:06.518843 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:06.519333 (+ 490us) server_negotiation.cc:299] Negotiation successful
0808 08:29:06.519451 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 08:29:06.520414 23928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:06.508311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:35779 (local address 127.22.235.195:47527)
0808 08:29:06.508792 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:06.508835 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:06.510014 (+ 1179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:06.510559 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:06.510567 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:06.510935 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:29:06.511475 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:06.511491 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:06.512353 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:06.512356 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:06.513375 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:06.513386 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:06.513943 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:06.513946 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:06.514105 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:06.514651 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:29:06.514668 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:29:06.516113 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:29:06.517816 (+ 1703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:29:06.517821 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:29:06.517833 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:29:06.518084 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:29:06.518347 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:29:06.518349 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:29:06.518351 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:29:06.518483 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:29:06.518857 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:29:06.518864 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:29:06.519106 (+ 242us) client_negotiation.cc:769] Sending connection context
0808 08:29:06.519272 (+ 166us) client_negotiation.cc:240] Negotiation successful
0808 08:29:06.519599 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":120,"threads_started":1}
I20260808 08:29:06.521569 23926 heartbeater.cc:347] Connected to a master server at 127.22.235.254:35779
I20260808 08:29:06.521785 23926 heartbeater.cc:464] Registering TS with master...
I20260808 08:29:06.522229 23926 heartbeater.cc:511] Master 127.22.235.254:35779 requested a full tablet report, sending...
I20260808 08:29:06.523267 23230 ts_manager.cc:194] Registered new tserver with Master: 88f23151d47a472681795eba527ee6f9 (127.22.235.195:39219)
I20260808 08:29:06.523813 23230 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.235.195@KRBTEST.COM'} at 127.22.235.195:47527
I20260808 08:29:06.525488 23471 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 08:29:06.534269 23285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:06.527447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59540 (local address 127.22.235.254:35779)
0808 08:29:06.527557 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:06.527561 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:06.527719 (+ 158us) server_negotiation.cc:407] Connection header received
0808 08:29:06.527834 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:06.527837 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:06.527880 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:06.527945 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:06.528641 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:06.529200 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:06.530004 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:06.530246 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:06.530333 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:06.531566 (+ 1233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:06.531570 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:06.531572 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:06.531597 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:06.532707 (+ 1110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:06.533100 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:06.533104 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:06.533105 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:06.533184 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:06.533430 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:06.533433 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:06.533435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:06.533658 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:06.533780 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:06.534008 (+ 228us) server_negotiation.cc:299] Negotiation successful
0808 08:29:06.534116 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 08:29:06.536904 23230 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59540:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 08:29:06.539817 23230 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 08:29:06.553699 23941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:06.548119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.195:39219 (local address 127.0.0.1:56898)
0808 08:29:06.548928 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:06.548941 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:06.549033 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:06.549637 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:06.549643 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:06.549667 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:29:06.549798 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:06.549804 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:06.550932 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:06.550936 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:06.552051 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:06.552063 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:06.553165 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:06.553168 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:06.553273 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:06.553309 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 08:29:06.553411 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 08:29:06.553542 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":746,"mutex_wait_us":1,"thread_start_us":126,"threads_started":1}
I20260808 08:29:06.553691 23944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:06.548321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56898 (local address 127.22.235.195:39219)
0808 08:29:06.548818 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:06.548833 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:06.549147 (+ 314us) server_negotiation.cc:407] Connection header received
0808 08:29:06.549236 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:06.549241 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:06.549465 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:06.549640 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:29:06.549917 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:06.550809 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:06.552197 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:06.553060 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:06.553166 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:06.553300 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:06.553454 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 08:29:06.553546 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":128,"threads_started":1}
I20260808 08:29:06.554210 23942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:06.548033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:45615 (local address 127.0.0.1:53790)
0808 08:29:06.548838 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:06.548887 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:06.549013 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:06.549946 (+ 933us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:06.549948 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:06.549971 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:29:06.550065 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:06.550071 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:06.551485 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:06.551496 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:06.552798 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:06.552810 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:06.553835 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:06.553839 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:06.553925 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:06.553947 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 08:29:06.554009 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 08:29:06.554076 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"mutex_wait_us":39,"thread_start_us":73,"threads_started":1}
I20260808 08:29:06.554355 23946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:06.548835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.194:40613 (local address 127.0.0.1:49480)
0808 08:29:06.549233 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:06.549245 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:06.549319 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:06.550143 (+ 824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:06.550146 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:06.550164 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:29:06.550260 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:06.550266 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:06.551706 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:06.551709 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:06.552752 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:06.552763 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:06.554005 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:06.554008 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:06.554098 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:06.554118 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 08:29:06.554188 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 08:29:06.554246 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"mutex_wait_us":20,"thread_start_us":91,"threads_started":1}
I20260808 08:29:06.554740 23947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:06.549086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49480 (local address 127.22.235.194:40613)
0808 08:29:06.549669 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:06.549681 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:06.549703 (+ 22us) server_negotiation.cc:407] Connection header received
0808 08:29:06.549774 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:06.549779 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:06.549964 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:06.550106 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:29:06.550373 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:06.551450 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:06.552866 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:06.553697 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:06.553820 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:06.553944 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:06.554240 (+ 296us) server_negotiation.cc:299] Negotiation successful
0808 08:29:06.554455 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":122,"threads_started":1}
I20260808 08:29:06.554821 23943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:06.548359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53790 (local address 127.22.235.193:45615)
0808 08:29:06.549509 (+ 1150us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:06.549524 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:06.549548 (+ 24us) server_negotiation.cc:407] Connection header received
0808 08:29:06.549618 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:06.549624 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:06.549796 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:06.549976 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:29:06.550197 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:06.551338 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:06.552904 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:06.553697 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:06.554409 (+ 712us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:06.554548 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:06.554602 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 08:29:06.554691 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1052,"thread_start_us":66,"threads_started":1}
I20260808 08:29:06.557111 23826 tablet_service.cc:1467] Processing CreateTablet for tablet 0a56c9c664d34804935c444c6e3949da (DEFAULT_TABLE table=test-table [id=b8abc67700ed48b18c663b0222f61253]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:29:06.557617 23411 tablet_service.cc:1467] Processing CreateTablet for tablet 0a56c9c664d34804935c444c6e3949da (DEFAULT_TABLE table=test-table [id=b8abc67700ed48b18c663b0222f61253]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:29:06.557613 23625 tablet_service.cc:1467] Processing CreateTablet for tablet 0a56c9c664d34804935c444c6e3949da (DEFAULT_TABLE table=test-table [id=b8abc67700ed48b18c663b0222f61253]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:29:06.558329 23826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a56c9c664d34804935c444c6e3949da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:29:06.558586 23411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a56c9c664d34804935c444c6e3949da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:29:06.558604 23625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a56c9c664d34804935c444c6e3949da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:29:06.564545 23950 tablet_bootstrap.cc:492] T 0a56c9c664d34804935c444c6e3949da P 79d414e06a8348ee87042b72995ed5eb: Bootstrap starting.
I20260808 08:29:06.566293 23954 tablet_bootstrap.cc:492] T 0a56c9c664d34804935c444c6e3949da P 88f23151d47a472681795eba527ee6f9: Bootstrap starting.
I20260808 08:29:06.566334 23951 tablet_bootstrap.cc:492] T 0a56c9c664d34804935c444c6e3949da P 92ba0a4bb75244baa0df2c1e6ad990c8: Bootstrap starting.
I20260808 08:29:06.568037 23950 tablet_bootstrap.cc:654] T 0a56c9c664d34804935c444c6e3949da P 79d414e06a8348ee87042b72995ed5eb: Neither blocks nor log segments found. Creating new log.
I20260808 08:29:06.568338 23954 tablet_bootstrap.cc:654] T 0a56c9c664d34804935c444c6e3949da P 88f23151d47a472681795eba527ee6f9: Neither blocks nor log segments found. Creating new log.
I20260808 08:29:06.568503 23951 tablet_bootstrap.cc:654] T 0a56c9c664d34804935c444c6e3949da P 92ba0a4bb75244baa0df2c1e6ad990c8: Neither blocks nor log segments found. Creating new log.
I20260808 08:29:06.569407 23954 log.cc:824] T 0a56c9c664d34804935c444c6e3949da P 88f23151d47a472681795eba527ee6f9: Log is configured to *not* fsync() on all Append() calls
I20260808 08:29:06.569448 23950 log.cc:824] T 0a56c9c664d34804935c444c6e3949da P 79d414e06a8348ee87042b72995ed5eb: Log is configured to *not* fsync() on all Append() calls
I20260808 08:29:06.569593 23951 log.cc:824] T 0a56c9c664d34804935c444c6e3949da P 92ba0a4bb75244baa0df2c1e6ad990c8: Log is configured to *not* fsync() on all Append() calls
I20260808 08:29:06.571376 23950 tablet_bootstrap.cc:492] T 0a56c9c664d34804935c444c6e3949da P 79d414e06a8348ee87042b72995ed5eb: No bootstrap required, opened a new log
I20260808 08:29:06.571375 23951 tablet_bootstrap.cc:492] T 0a56c9c664d34804935c444c6e3949da P 92ba0a4bb75244baa0df2c1e6ad990c8: No bootstrap required, opened a new log
I20260808 08:29:06.571393 23954 tablet_bootstrap.cc:492] T 0a56c9c664d34804935c444c6e3949da P 88f23151d47a472681795eba527ee6f9: No bootstrap required, opened a new log
I20260808 08:29:06.571583 23950 ts_tablet_manager.cc:1397] T 0a56c9c664d34804935c444c6e3949da P 79d414e06a8348ee87042b72995ed5eb: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260808 08:29:06.571584 23954 ts_tablet_manager.cc:1397] T 0a56c9c664d34804935c444c6e3949da P 88f23151d47a472681795eba527ee6f9: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260808 08:29:06.571605 23951 ts_tablet_manager.cc:1397] T 0a56c9c664d34804935c444c6e3949da P 92ba0a4bb75244baa0df2c1e6ad990c8: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 08:29:06.574375 23954 raft_consensus.cc:348] T 0a56c9c664d34804935c444c6e3949da P 88f23151d47a472681795eba527ee6f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79d414e06a8348ee87042b72995ed5eb" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45615 } } peers { permanent_uuid: "88f23151d47a472681795eba527ee6f9" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39219 } } peers { permanent_uuid: "92ba0a4bb75244baa0df2c1e6ad990c8" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 40613 } }
I20260808 08:29:06.574594 23954 raft_consensus.cc:374] T 0a56c9c664d34804935c444c6e3949da P 88f23151d47a472681795eba527ee6f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:29:06.574651 23954 raft_consensus.cc:729] T 0a56c9c664d34804935c444c6e3949da P 88f23151d47a472681795eba527ee6f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88f23151d47a472681795eba527ee6f9, State: Initialized, Role: FOLLOWER
I20260808 08:29:06.575043 23954 consensus_queue.cc:260] T 0a56c9c664d34804935c444c6e3949da P 88f23151d47a472681795eba527ee6f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79d414e06a8348ee87042b72995ed5eb" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45615 } } peers { permanent_uuid: "88f23151d47a472681795eba527ee6f9" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39219 } } peers { permanent_uuid: "92ba0a4bb75244baa0df2c1e6ad990c8" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 40613 } }
I20260808 08:29:06.575296 23951 raft_consensus.cc:348] T 0a56c9c664d34804935c444c6e3949da P 92ba0a4bb75244baa0df2c1e6ad990c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79d414e06a8348ee87042b72995ed5eb" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45615 } } peers { permanent_uuid: "88f23151d47a472681795eba527ee6f9" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39219 } } peers { permanent_uuid: "92ba0a4bb75244baa0df2c1e6ad990c8" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 40613 } }
I20260808 08:29:06.575296 23950 raft_consensus.cc:348] T 0a56c9c664d34804935c444c6e3949da P 79d414e06a8348ee87042b72995ed5eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79d414e06a8348ee87042b72995ed5eb" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45615 } } peers { permanent_uuid: "88f23151d47a472681795eba527ee6f9" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39219 } } peers { permanent_uuid: "92ba0a4bb75244baa0df2c1e6ad990c8" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 40613 } }
I20260808 08:29:06.575546 23950 raft_consensus.cc:374] T 0a56c9c664d34804935c444c6e3949da P 79d414e06a8348ee87042b72995ed5eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:29:06.575546 23951 raft_consensus.cc:374] T 0a56c9c664d34804935c444c6e3949da P 92ba0a4bb75244baa0df2c1e6ad990c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:29:06.575601 23950 raft_consensus.cc:729] T 0a56c9c664d34804935c444c6e3949da P 79d414e06a8348ee87042b72995ed5eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79d414e06a8348ee87042b72995ed5eb, State: Initialized, Role: FOLLOWER
I20260808 08:29:06.575601 23951 raft_consensus.cc:729] T 0a56c9c664d34804935c444c6e3949da P 92ba0a4bb75244baa0df2c1e6ad990c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92ba0a4bb75244baa0df2c1e6ad990c8, State: Initialized, Role: FOLLOWER
I20260808 08:29:06.575750 23926 heartbeater.cc:503] Master 127.22.235.254:35779 was elected leader, sending a full tablet report...
I20260808 08:29:06.576022 23954 ts_tablet_manager.cc:1428] T 0a56c9c664d34804935c444c6e3949da P 88f23151d47a472681795eba527ee6f9: Time spent starting tablet: real 0.004s user 0.003s sys 0.002s
I20260808 08:29:06.576022 23951 consensus_queue.cc:260] T 0a56c9c664d34804935c444c6e3949da P 92ba0a4bb75244baa0df2c1e6ad990c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79d414e06a8348ee87042b72995ed5eb" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45615 } } peers { permanent_uuid: "88f23151d47a472681795eba527ee6f9" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39219 } } peers { permanent_uuid: "92ba0a4bb75244baa0df2c1e6ad990c8" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 40613 } }
I20260808 08:29:06.576043 23950 consensus_queue.cc:260] T 0a56c9c664d34804935c444c6e3949da P 79d414e06a8348ee87042b72995ed5eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79d414e06a8348ee87042b72995ed5eb" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45615 } } peers { permanent_uuid: "88f23151d47a472681795eba527ee6f9" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39219 } } peers { permanent_uuid: "92ba0a4bb75244baa0df2c1e6ad990c8" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 40613 } }
I20260808 08:29:06.576591 23516 heartbeater.cc:503] Master 127.22.235.254:35779 was elected leader, sending a full tablet report...
I20260808 08:29:06.576962 23951 ts_tablet_manager.cc:1428] T 0a56c9c664d34804935c444c6e3949da P 92ba0a4bb75244baa0df2c1e6ad990c8: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 08:29:06.576962 23950 ts_tablet_manager.cc:1428] T 0a56c9c664d34804935c444c6e3949da P 79d414e06a8348ee87042b72995ed5eb: Time spent starting tablet: real 0.005s user 0.004s sys 0.003s
I20260808 08:29:06.577324 23728 heartbeater.cc:503] Master 127.22.235.254:35779 was elected leader, sending a full tablet report...
I20260808 08:29:06.659572 23964 raft_consensus.cc:482] T 0a56c9c664d34804935c444c6e3949da P 92ba0a4bb75244baa0df2c1e6ad990c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 08:29:06.659838 23964 raft_consensus.cc:504] T 0a56c9c664d34804935c444c6e3949da P 92ba0a4bb75244baa0df2c1e6ad990c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79d414e06a8348ee87042b72995ed5eb" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45615 } } peers { permanent_uuid: "88f23151d47a472681795eba527ee6f9" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39219 } } peers { permanent_uuid: "92ba0a4bb75244baa0df2c1e6ad990c8" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 40613 } }
I20260808 08:29:06.661262 23964 leader_election.cc:288] T 0a56c9c664d34804935c444c6e3949da P 92ba0a4bb75244baa0df2c1e6ad990c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79d414e06a8348ee87042b72995ed5eb (127.22.235.193:45615), 88f23151d47a472681795eba527ee6f9 (127.22.235.195:39219)
I20260808 08:29:06.664783 23730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:06.661501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:45615 (local address 127.22.235.194:34951)
0808 08:29:06.661611 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:06.661625 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:06.661726 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:06.662048 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:06.662051 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:06.662071 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:29:06.662192 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:06.662197 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:06.662913 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:06.662916 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:06.663691 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:06.663700 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:06.664332 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:06.664336 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:06.664440 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:06.664459 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 08:29:06.664532 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 08:29:06.664650 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"mutex_wait_us":1,"spinlock_wait_cycles":12544}
I20260808 08:29:06.664780 23943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:06.661640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:34951 (local address 127.22.235.193:45615)
0808 08:29:06.661772 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:06.661777 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:06.661794 (+ 17us) server_negotiation.cc:407] Connection header received
0808 08:29:06.661857 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:06.661860 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:06.661926 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:06.662024 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:29:06.662300 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:06.662802 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:06.663818 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:06.664219 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:06.664327 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:06.664384 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:06.664560 (+ 176us) server_negotiation.cc:299] Negotiation successful
0808 08:29:06.664659 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 08:29:06.665001 23944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:06.661714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:37911 (local address 127.22.235.195:39219)
0808 08:29:06.661844 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:06.661848 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:06.662093 (+ 245us) server_negotiation.cc:407] Connection header received
0808 08:29:06.662202 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:06.662206 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:06.662253 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:06.662314 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:29:06.662591 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:06.663081 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:06.664064 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:06.664444 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:06.664547 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:06.664594 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:06.664820 (+ 226us) server_negotiation.cc:299] Negotiation successful
0808 08:29:06.664886 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 08:29:06.665010 23965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:06.661506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.195:39219 (local address 127.22.235.194:37911)
0808 08:29:06.662005 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:06.662016 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:06.662117 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:06.662336 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:06.662339 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:06.662355 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:29:06.662481 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:06.662486 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:06.663187 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:06.663190 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:06.663947 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:06.663955 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:06.664557 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:06.664561 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:06.664711 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:06.664729 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 08:29:06.664795 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 08:29:06.664859 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"mutex_wait_us":187,"thread_start_us":149,"threads_started":1}
I20260808 08:29:06.665752 23865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a56c9c664d34804935c444c6e3949da" candidate_uuid: "92ba0a4bb75244baa0df2c1e6ad990c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88f23151d47a472681795eba527ee6f9" is_pre_election: true
I20260808 08:29:06.665755 23444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a56c9c664d34804935c444c6e3949da" candidate_uuid: "92ba0a4bb75244baa0df2c1e6ad990c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79d414e06a8348ee87042b72995ed5eb" is_pre_election: true
I20260808 08:29:06.666036 23444 raft_consensus.cc:2454] T 0a56c9c664d34804935c444c6e3949da P 79d414e06a8348ee87042b72995ed5eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92ba0a4bb75244baa0df2c1e6ad990c8 in term 0.
I20260808 08:29:06.666036 23865 raft_consensus.cc:2454] T 0a56c9c664d34804935c444c6e3949da P 88f23151d47a472681795eba527ee6f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92ba0a4bb75244baa0df2c1e6ad990c8 in term 0.
I20260808 08:29:06.666507 23559 leader_election.cc:302] T 0a56c9c664d34804935c444c6e3949da P 92ba0a4bb75244baa0df2c1e6ad990c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88f23151d47a472681795eba527ee6f9, 92ba0a4bb75244baa0df2c1e6ad990c8; no voters:
I20260808 08:29:06.666833 23964 raft_consensus.cc:2781] T 0a56c9c664d34804935c444c6e3949da P 92ba0a4bb75244baa0df2c1e6ad990c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 08:29:06.666914 23964 raft_consensus.cc:482] T 0a56c9c664d34804935c444c6e3949da P 92ba0a4bb75244baa0df2c1e6ad990c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 08:29:06.666947 23964 raft_consensus.cc:3037] T 0a56c9c664d34804935c444c6e3949da P 92ba0a4bb75244baa0df2c1e6ad990c8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:29:06.668174 23964 raft_consensus.cc:504] T 0a56c9c664d34804935c444c6e3949da P 92ba0a4bb75244baa0df2c1e6ad990c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79d414e06a8348ee87042b72995ed5eb" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45615 } } peers { permanent_uuid: "88f23151d47a472681795eba527ee6f9" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39219 } } peers { permanent_uuid: "92ba0a4bb75244baa0df2c1e6ad990c8" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 40613 } }
I20260808 08:29:06.668514 23964 leader_election.cc:288] T 0a56c9c664d34804935c444c6e3949da P 92ba0a4bb75244baa0df2c1e6ad990c8 [CANDIDATE]: Term 1 election: Requested vote from peers 79d414e06a8348ee87042b72995ed5eb (127.22.235.193:45615), 88f23151d47a472681795eba527ee6f9 (127.22.235.195:39219)
I20260808 08:29:06.668855 23865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a56c9c664d34804935c444c6e3949da" candidate_uuid: "92ba0a4bb75244baa0df2c1e6ad990c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88f23151d47a472681795eba527ee6f9"
I20260808 08:29:06.668869 23444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a56c9c664d34804935c444c6e3949da" candidate_uuid: "92ba0a4bb75244baa0df2c1e6ad990c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79d414e06a8348ee87042b72995ed5eb"
I20260808 08:29:06.668987 23444 raft_consensus.cc:3037] T 0a56c9c664d34804935c444c6e3949da P 79d414e06a8348ee87042b72995ed5eb [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:29:06.668998 23865 raft_consensus.cc:3037] T 0a56c9c664d34804935c444c6e3949da P 88f23151d47a472681795eba527ee6f9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:29:06.670038 23865 raft_consensus.cc:2454] T 0a56c9c664d34804935c444c6e3949da P 88f23151d47a472681795eba527ee6f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92ba0a4bb75244baa0df2c1e6ad990c8 in term 1.
I20260808 08:29:06.670104 23444 raft_consensus.cc:2454] T 0a56c9c664d34804935c444c6e3949da P 79d414e06a8348ee87042b72995ed5eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92ba0a4bb75244baa0df2c1e6ad990c8 in term 1.
I20260808 08:29:06.670336 23559 leader_election.cc:302] T 0a56c9c664d34804935c444c6e3949da P 92ba0a4bb75244baa0df2c1e6ad990c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88f23151d47a472681795eba527ee6f9, 92ba0a4bb75244baa0df2c1e6ad990c8; no voters:
I20260808 08:29:06.670497 23964 raft_consensus.cc:2781] T 0a56c9c664d34804935c444c6e3949da P 92ba0a4bb75244baa0df2c1e6ad990c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:29:06.670701 23964 raft_consensus.cc:686] T 0a56c9c664d34804935c444c6e3949da P 92ba0a4bb75244baa0df2c1e6ad990c8 [term 1 LEADER]: Becoming Leader. State: Replica: 92ba0a4bb75244baa0df2c1e6ad990c8, State: Running, Role: LEADER
I20260808 08:29:06.671008 23964 consensus_queue.cc:237] T 0a56c9c664d34804935c444c6e3949da P 92ba0a4bb75244baa0df2c1e6ad990c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79d414e06a8348ee87042b72995ed5eb" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45615 } } peers { permanent_uuid: "88f23151d47a472681795eba527ee6f9" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39219 } } peers { permanent_uuid: "92ba0a4bb75244baa0df2c1e6ad990c8" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 40613 } }
I20260808 08:29:06.674126 23224 catalog_manager.cc:5495] T 0a56c9c664d34804935c444c6e3949da P 92ba0a4bb75244baa0df2c1e6ad990c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 92ba0a4bb75244baa0df2c1e6ad990c8 (127.22.235.194). New cstate: current_term: 1 leader_uuid: "92ba0a4bb75244baa0df2c1e6ad990c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79d414e06a8348ee87042b72995ed5eb" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88f23151d47a472681795eba527ee6f9" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92ba0a4bb75244baa0df2c1e6ad990c8" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 40613 } health_report { overall_health: HEALTHY } } }
I20260808 08:29:06.706607 23947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:06.703230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49488 (local address 127.22.235.194:40613)
0808 08:29:06.703354 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:06.703359 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:06.703376 (+ 17us) server_negotiation.cc:407] Connection header received
0808 08:29:06.703456 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:06.703458 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:06.703517 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:06.703610 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:29:06.703836 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:06.704338 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:06.705238 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:06.705445 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:06.705533 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:06.705827 (+ 294us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:29:06.706241 (+ 414us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 08:29:06.706348 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:06.706418 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 08:29:06.706475 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 08:29:06.712323 23865 raft_consensus.cc:1261] T 0a56c9c664d34804935c444c6e3949da P 88f23151d47a472681795eba527ee6f9 [term 1 FOLLOWER]: Refusing update from remote peer 92ba0a4bb75244baa0df2c1e6ad990c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:29:06.712324 23444 raft_consensus.cc:1261] T 0a56c9c664d34804935c444c6e3949da P 79d414e06a8348ee87042b72995ed5eb [term 1 FOLLOWER]: Refusing update from remote peer 92ba0a4bb75244baa0df2c1e6ad990c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:29:06.713039 23964 consensus_queue.cc:1035] T 0a56c9c664d34804935c444c6e3949da P 92ba0a4bb75244baa0df2c1e6ad990c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88f23151d47a472681795eba527ee6f9" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 39219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:29:06.713215 23966 consensus_queue.cc:1035] T 0a56c9c664d34804935c444c6e3949da P 92ba0a4bb75244baa0df2c1e6ad990c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79d414e06a8348ee87042b72995ed5eb" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 45615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:29:06.720374 23975 mvcc.cc:205] Tried to move back new op lower bound from 7316184050526601216 to 7316184050366799872. Current Snapshot: MvccSnapshot[applied={T|T < 7316184050526601216}]
I20260808 08:29:06.720644 23973 mvcc.cc:205] Tried to move back new op lower bound from 7316184050526601216 to 7316184050366799872. Current Snapshot: MvccSnapshot[applied={T|T < 7316184050526601216}]
I20260808 08:29:06.728108 23224 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59540:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:29:06.728252 23224 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59540:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:29:06.730700 23224 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2985a84ac7d242e6806ad1c4ab0452c4: Sending DeleteTablet for 3 replicas of tablet 0a56c9c664d34804935c444c6e3949da
I20260808 08:29:06.731359 23411 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a56c9c664d34804935c444c6e3949da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:29:06 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:53790
I20260808 08:29:06.731577 23826 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a56c9c664d34804935c444c6e3949da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:29:06 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:56898
I20260808 08:29:06.731685 23625 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a56c9c664d34804935c444c6e3949da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:29:06 UTC) from {username='kudu', principal='kudu/127.22.235.254@KRBTEST.COM'} at 127.0.0.1:49480
I20260808 08:29:06.731983 23989 tablet_replica.cc:330] T 0a56c9c664d34804935c444c6e3949da P 88f23151d47a472681795eba527ee6f9: stopping tablet replica
I20260808 08:29:06.731983 23988 tablet_replica.cc:330] T 0a56c9c664d34804935c444c6e3949da P 79d414e06a8348ee87042b72995ed5eb: stopping tablet replica
I20260808 08:29:06.732275 23988 raft_consensus.cc:2229] T 0a56c9c664d34804935c444c6e3949da P 79d414e06a8348ee87042b72995ed5eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 08:29:06.732275 23989 raft_consensus.cc:2229] T 0a56c9c664d34804935c444c6e3949da P 88f23151d47a472681795eba527ee6f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 08:29:06.732568 23988 raft_consensus.cc:2258] T 0a56c9c664d34804935c444c6e3949da P 79d414e06a8348ee87042b72995ed5eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:29:06.732568 23989 raft_consensus.cc:2258] T 0a56c9c664d34804935c444c6e3949da P 88f23151d47a472681795eba527ee6f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:29:06.733012 23990 tablet_replica.cc:330] T 0a56c9c664d34804935c444c6e3949da P 92ba0a4bb75244baa0df2c1e6ad990c8: stopping tablet replica
I20260808 08:29:06.733263 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 23317
I20260808 08:29:06.733268 23990 raft_consensus.cc:2229] T 0a56c9c664d34804935c444c6e3949da P 92ba0a4bb75244baa0df2c1e6ad990c8 [term 1 LEADER]: Raft consensus shutting down.
I20260808 08:29:06.733626 23990 raft_consensus.cc:2258] T 0a56c9c664d34804935c444c6e3949da P 92ba0a4bb75244baa0df2c1e6ad990c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:29:06.734104 23989 ts_tablet_manager.cc:1905] T 0a56c9c664d34804935c444c6e3949da P 88f23151d47a472681795eba527ee6f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:29:06.736390 23989 ts_tablet_manager.cc:1918] T 0a56c9c664d34804935c444c6e3949da P 88f23151d47a472681795eba527ee6f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 08:29:06.736475 23989 log.cc:1196] T 0a56c9c664d34804935c444c6e3949da P 88f23151d47a472681795eba527ee6f9: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-2/wal/wals/0a56c9c664d34804935c444c6e3949da
I20260808 08:29:06.736819 23989 ts_tablet_manager.cc:1939] T 0a56c9c664d34804935c444c6e3949da P 88f23151d47a472681795eba527ee6f9: Deleting consensus metadata
I20260808 08:29:06.737404 23990 ts_tablet_manager.cc:1905] T 0a56c9c664d34804935c444c6e3949da P 92ba0a4bb75244baa0df2c1e6ad990c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:29:06.739449 23990 ts_tablet_manager.cc:1918] T 0a56c9c664d34804935c444c6e3949da P 92ba0a4bb75244baa0df2c1e6ad990c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 08:29:06.739513 23990 log.cc:1196] T 0a56c9c664d34804935c444c6e3949da P 92ba0a4bb75244baa0df2c1e6ad990c8: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786177493204855-23471-0/minicluster-data/ts-1/wal/wals/0a56c9c664d34804935c444c6e3949da
I20260808 08:29:06.739694 23990 ts_tablet_manager.cc:1939] T 0a56c9c664d34804935c444c6e3949da P 92ba0a4bb75244baa0df2c1e6ad990c8: Deleting consensus metadata
I20260808 08:29:06.742056 23211 catalog_manager.cc:4841] TS 88f23151d47a472681795eba527ee6f9 (127.22.235.195:39219): tablet 0a56c9c664d34804935c444c6e3949da (table test-table [id=b8abc67700ed48b18c663b0222f61253]) successfully deleted
I20260808 08:29:06.742254 23211 catalog_manager.cc:4841] TS 92ba0a4bb75244baa0df2c1e6ad990c8 (127.22.235.194:40613): tablet 0a56c9c664d34804935c444c6e3949da (table test-table [id=b8abc67700ed48b18c663b0222f61253]) successfully deleted
W20260808 08:29:06.750530 23207 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.235.193:45615 (error 108)
I20260808 08:29:06.750802 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 23523
I20260808 08:29:06.753764 23941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:06.753390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:45615 (local address 127.0.0.1:53806)
0808 08:29:06.753601 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:06.753678 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.193:45615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
W20260808 08:29:06.753949 23207 catalog_manager.cc:4584] TS 79d414e06a8348ee87042b72995ed5eb (127.22.235.193:45615): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0a56c9c664d34804935c444c6e3949da: Network error: Client connection negotiation failed: client connection to 127.22.235.193:45615: connect: Connection refused (error 111)
I20260808 08:29:06.785647 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 23733
I20260808 08:29:06.801908 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 23173
2026-08-08T08:29:06Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2524 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-08T08:29:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:29:06Z Disabled control of system clock
I20260808 08:29:06.839311 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:33665
--webserver_interface=127.22.235.254
--webserver_port=0
--builtin_ntp_servers=127.22.235.212:35321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:33665
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 08:29:06.952790 23996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:29:06.953053 23996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:29:06.953086 23996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:29:06.957057 23996 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:29:06.957108 23996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:29:06.957122 23996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:29:06.957137 23996 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:29:06.957154 23996 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:29:06.961790 23996 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:35321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:33665
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:33665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.235.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23996
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:29:06.962823 23996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:29:06.964146 23996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:29:06.972524 24005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:06.972555 24007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:06.972565 24004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:06.972898 23996 server_base.cc:1034] running on GCE node
I20260808 08:29:06.973461 23996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:29:06.974524 23996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:29:06.975682 23996 hybrid_clock.cc:648] HybridClock initialized: now 1786177746975669 us; error 25 us; skew 500 ppm
I20260808 08:29:06.983028 23996 webserver.cc:460] Webserver started at http://127.22.235.254:37357/ using document root <none> and password file <none>
I20260808 08:29:06.983604 23996 fs_manager.cc:359] Metadata directory not provided
I20260808 08:29:06.983672 23996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:29:06.983873 23996 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:29:06.985958 23996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "7f89304a85b4432d909a236f5874d142"
format_stamp: "Formatted at 2026-08-08 08:29:06 on dist-test-slave-f7th"
server_key: "643ad16a654fdd85dd3345ee986214ef"
server_key_iv: "2e1d7e4a0a22900c0379b2da22682ec3"
server_key_version: "encryptionkey@0"
I20260808 08:29:06.986500 23996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "7f89304a85b4432d909a236f5874d142"
format_stamp: "Formatted at 2026-08-08 08:29:06 on dist-test-slave-f7th"
server_key: "643ad16a654fdd85dd3345ee986214ef"
server_key_iv: "2e1d7e4a0a22900c0379b2da22682ec3"
server_key_version: "encryptionkey@0"
I20260808 08:29:06.990147 23996 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 08:29:06.992601 24017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:06.993782 23996 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:29:06.993901 23996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "7f89304a85b4432d909a236f5874d142"
format_stamp: "Formatted at 2026-08-08 08:29:06 on dist-test-slave-f7th"
server_key: "643ad16a654fdd85dd3345ee986214ef"
server_key_iv: "2e1d7e4a0a22900c0379b2da22682ec3"
server_key_version: "encryptionkey@0"
I20260808 08:29:06.993996 23996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:29:07.007356 23996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:29:07.008113 23996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:29:07.008298 23996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:29:07.015532 23996 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.254:33665
I20260808 08:29:07.015548 24092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.254:33665 every 8 connection(s)
I20260808 08:29:07.016495 23996 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:29:07.019698 24093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:29:07.025261 24093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f89304a85b4432d909a236f5874d142: Bootstrap starting.
I20260808 08:29:07.025295 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 23996
I20260808 08:29:07.025401 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:29:07.025656 23471 external_mini_cluster.cc:1428] Setting key 4e10fb404f65f7aff7196fc4b2483ec5
I20260808 08:29:07.027760 24093 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f89304a85b4432d909a236f5874d142: Neither blocks nor log segments found. Creating new log.
I20260808 08:29:07.028512 24093 log.cc:824] T 00000000000000000000000000000000 P 7f89304a85b4432d909a236f5874d142: Log is configured to *not* fsync() on all Append() calls
I20260808 08:29:07.030367 24093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f89304a85b4432d909a236f5874d142: No bootstrap required, opened a new log
I20260808 08:29:07.034018 24093 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f89304a85b4432d909a236f5874d142 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f89304a85b4432d909a236f5874d142" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 33665 } }
I20260808 08:29:07.034243 24093 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7f89304a85b4432d909a236f5874d142 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:29:07.034301 24093 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f89304a85b4432d909a236f5874d142 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f89304a85b4432d909a236f5874d142, State: Initialized, Role: FOLLOWER
I20260808 08:29:07.034731 24098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:07.027040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43356 (local address 127.22.235.254:33665)
0808 08:29:07.027647 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:07.027660 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:07.027696 (+ 36us) server_negotiation.cc:407] Connection header received
0808 08:29:07.028337 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:07.028355 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:07.028799 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:07.029099 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:07.029534 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:07.030525 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:07.031412 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:07.031886 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:07.032043 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:07.032930 (+ 887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:07.032935 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:07.032948 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:29:07.033320 (+ 372us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:07.033408 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:29:07.033431 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:07.033545 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:07.033681 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 08:29:07.033933 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":110,"threads_started":1}
I20260808 08:29:07.034711 24093 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f89304a85b4432d909a236f5874d142 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f89304a85b4432d909a236f5874d142" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 33665 } }
I20260808 08:29:07.034842 24093 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7f89304a85b4432d909a236f5874d142 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:29:07.034915 24093 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f89304a85b4432d909a236f5874d142 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:29:07.035010 24093 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f89304a85b4432d909a236f5874d142 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:29:07.035924 24093 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f89304a85b4432d909a236f5874d142 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f89304a85b4432d909a236f5874d142" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 33665 } }
I20260808 08:29:07.036295 24093 leader_election.cc:302] T 00000000000000000000000000000000 P 7f89304a85b4432d909a236f5874d142 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f89304a85b4432d909a236f5874d142; no voters:
I20260808 08:29:07.036656 24093 leader_election.cc:288] T 00000000000000000000000000000000 P 7f89304a85b4432d909a236f5874d142 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:29:07.036744 24104 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f89304a85b4432d909a236f5874d142 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:29:07.036966 24104 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f89304a85b4432d909a236f5874d142 [term 1 LEADER]: Becoming Leader. State: Replica: 7f89304a85b4432d909a236f5874d142, State: Running, Role: LEADER
I20260808 08:29:07.037267 24104 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f89304a85b4432d909a236f5874d142 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f89304a85b4432d909a236f5874d142" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 33665 } }
I20260808 08:29:07.037864 24093 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f89304a85b4432d909a236f5874d142 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:29:07.038856 24109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f89304a85b4432d909a236f5874d142 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f89304a85b4432d909a236f5874d142. Latest consensus state: current_term: 1 leader_uuid: "7f89304a85b4432d909a236f5874d142" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f89304a85b4432d909a236f5874d142" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 33665 } } }
I20260808 08:29:07.039129 24109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f89304a85b4432d909a236f5874d142 [sys.catalog]: This master's current role is: LEADER
I20260808 08:29:07.038856 24108 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f89304a85b4432d909a236f5874d142 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7f89304a85b4432d909a236f5874d142" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f89304a85b4432d909a236f5874d142" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 33665 } } }
I20260808 08:29:07.039273 24108 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f89304a85b4432d909a236f5874d142 [sys.catalog]: This master's current role is: LEADER
I20260808 08:29:07.040081 24117 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:29:07.042791 24117 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:29:07.048481 24117 catalog_manager.cc:1347] Generated new cluster ID: e1f915a93f0348ceb3a03b8abdc195bc
I20260808 08:29:07.048538 24117 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:29:07.064071 24117 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:29:07.065291 24117 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:29:07.072894 24117 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f89304a85b4432d909a236f5874d142: Generated new TSK 0
I20260808 08:29:07.073673 24117 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 08:29:07.083395 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.193:0
--local_ip_for_outbound_sockets=127.22.235.193
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:33665
--builtin_ntp_servers=127.22.235.212:35321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 08:29:07.195941 24131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:29:07.196195 24131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:29:07.196247 24131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:29:07.200214 24131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:29:07.200268 24131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:29:07.200340 24131 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.193
I20260808 08:29:07.204892 24131 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:35321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.235.193
--webserver_port=0
--tserver_master_addrs=127.22.235.254:33665
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24131
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.193
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:29:07.205878 24131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:29:07.207214 24131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:29:07.216408 24140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:07.216425 24131 server_base.cc:1034] running on GCE node
W20260808 08:29:07.216449 24144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:07.216425 24141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:07.217039 24131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:29:07.217670 24131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:29:07.218871 24131 hybrid_clock.cc:648] HybridClock initialized: now 1786177747218853 us; error 25 us; skew 500 ppm
I20260808 08:29:07.220701 24131 webserver.cc:460] Webserver started at http://127.22.235.193:45613/ using document root <none> and password file <none>
I20260808 08:29:07.221287 24131 fs_manager.cc:359] Metadata directory not provided
I20260808 08:29:07.221345 24131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:29:07.221544 24131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:29:07.223541 24131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-0/data/instance:
uuid: "dd5cc9b2cda24b3f96795f300c42b43e"
format_stamp: "Formatted at 2026-08-08 08:29:07 on dist-test-slave-f7th"
server_key: "0fb065912015f137892410d93cc47aa7"
server_key_iv: "558897b69c5cc2965e6e07588e47f071"
server_key_version: "encryptionkey@0"
I20260808 08:29:07.224064 24131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance:
uuid: "dd5cc9b2cda24b3f96795f300c42b43e"
format_stamp: "Formatted at 2026-08-08 08:29:07 on dist-test-slave-f7th"
server_key: "0fb065912015f137892410d93cc47aa7"
server_key_iv: "558897b69c5cc2965e6e07588e47f071"
server_key_version: "encryptionkey@0"
I20260808 08:29:07.227612 24131 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260808 08:29:07.229987 24151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:07.231170 24131 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 08:29:07.231283 24131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-0/wal
uuid: "dd5cc9b2cda24b3f96795f300c42b43e"
format_stamp: "Formatted at 2026-08-08 08:29:07 on dist-test-slave-f7th"
server_key: "0fb065912015f137892410d93cc47aa7"
server_key_iv: "558897b69c5cc2965e6e07588e47f071"
server_key_version: "encryptionkey@0"
I20260808 08:29:07.231374 24131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:29:07.258999 24131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:29:07.259768 24131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:29:07.259954 24131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:29:07.260603 24131 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:29:07.261632 24131 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:29:07.261684 24131 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:07.261739 24131 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:29:07.261770 24131 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:07.269869 24131 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.193:38733
I20260808 08:29:07.269894 24321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.193:38733 every 8 connection(s)
I20260808 08:29:07.270742 24131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-0/data/info.pb
I20260808 08:29:07.279443 24098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:07.272479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.193:41705 (local address 127.22.235.254:33665)
0808 08:29:07.272609 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:07.272632 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:07.273680 (+ 1048us) server_negotiation.cc:407] Connection header received
0808 08:29:07.275044 (+ 1364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:07.275048 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:07.275106 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:07.275178 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:07.276043 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:07.276565 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:07.277704 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:07.277923 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:07.278023 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:07.278996 (+ 973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:07.278999 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:07.279001 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:29:07.279013 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:07.279031 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:29:07.279039 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:07.279096 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:07.279238 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 08:29:07.279297 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 08:29:07.280249 24327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:07.272896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:33665 (local address 127.22.235.193:41705)
0808 08:29:07.273542 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:07.273579 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:07.274823 (+ 1244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:07.275351 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:07.275358 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:07.275438 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:29:07.275904 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:07.275916 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:07.276729 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:07.276732 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:07.277584 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:07.277594 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:07.278067 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:07.278070 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:07.278232 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:07.278748 (+ 516us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 08:29:07.278749 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:29:07.278787 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 08:29:07.278848 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:29:07.279107 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:29:07.279116 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:29:07.279123 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 08:29:07.279227 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 08:29:07.279487 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":121,"threads_started":1}
I20260808 08:29:07.280750 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 24131
I20260808 08:29:07.280860 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-0/wal/instance
I20260808 08:29:07.281133 23471 external_mini_cluster.cc:1428] Setting key 259a4fbb0a3fdb1da30e3af316ee508d
I20260808 08:29:07.281430 24325 heartbeater.cc:347] Connected to a master server at 127.22.235.254:33665
I20260808 08:29:07.281672 24325 heartbeater.cc:464] Registering TS with master...
I20260808 08:29:07.282186 24325 heartbeater.cc:511] Master 127.22.235.254:33665 requested a full tablet report, sending...
I20260808 08:29:07.282965 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.194:0
--local_ip_for_outbound_sockets=127.22.235.194
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:33665
--builtin_ntp_servers=127.22.235.212:35321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 08:29:07.283668 24038 ts_manager.cc:194] Registered new tserver with Master: dd5cc9b2cda24b3f96795f300c42b43e (127.22.235.193:38733)
I20260808 08:29:07.284902 24038 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.235.193:41705
W20260808 08:29:07.397379 24328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:29:07.397635 24328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:29:07.397670 24328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:29:07.401593 24328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:29:07.401645 24328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:29:07.401695 24328 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.194
I20260808 08:29:07.406348 24328 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:35321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.235.194
--webserver_port=0
--tserver_master_addrs=127.22.235.254:33665
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.194
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:29:07.407467 24328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:29:07.408856 24328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:29:07.418332 24342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:07.418339 24341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:07.418385 24347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:07.418615 24328 server_base.cc:1034] running on GCE node
I20260808 08:29:07.419099 24328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:29:07.419755 24328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:29:07.420938 24328 hybrid_clock.cc:648] HybridClock initialized: now 1786177747420915 us; error 30 us; skew 500 ppm
I20260808 08:29:07.422850 24328 webserver.cc:460] Webserver started at http://127.22.235.194:44245/ using document root <none> and password file <none>
I20260808 08:29:07.423416 24328 fs_manager.cc:359] Metadata directory not provided
I20260808 08:29:07.423480 24328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:29:07.423686 24328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:29:07.425755 24328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-1/data/instance:
uuid: "abaaac2a861a43ea806ba70a05fe357e"
format_stamp: "Formatted at 2026-08-08 08:29:07 on dist-test-slave-f7th"
server_key: "d67b98ae64cd9be96acb9723d6426879"
server_key_iv: "df739dfc3490f78a83807cb5b5002a30"
server_key_version: "encryptionkey@0"
I20260808 08:29:07.426285 24328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance:
uuid: "abaaac2a861a43ea806ba70a05fe357e"
format_stamp: "Formatted at 2026-08-08 08:29:07 on dist-test-slave-f7th"
server_key: "d67b98ae64cd9be96acb9723d6426879"
server_key_iv: "df739dfc3490f78a83807cb5b5002a30"
server_key_version: "encryptionkey@0"
I20260808 08:29:07.429870 24328 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 08:29:07.432224 24357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:07.433420 24328 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:29:07.433547 24328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-1/wal
uuid: "abaaac2a861a43ea806ba70a05fe357e"
format_stamp: "Formatted at 2026-08-08 08:29:07 on dist-test-slave-f7th"
server_key: "d67b98ae64cd9be96acb9723d6426879"
server_key_iv: "df739dfc3490f78a83807cb5b5002a30"
server_key_version: "encryptionkey@0"
I20260808 08:29:07.433636 24328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:29:07.448722 24328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:29:07.449335 24328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:29:07.449508 24328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:29:07.450048 24328 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:29:07.450979 24328 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:29:07.451028 24328 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:07.451081 24328 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:29:07.451107 24328 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:07.459396 24328 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.194:44061
I20260808 08:29:07.459518 24530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.194:44061 every 8 connection(s)
I20260808 08:29:07.460265 24328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-1/data/info.pb
I20260808 08:29:07.461053 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 24328
I20260808 08:29:07.461169 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-1/wal/instance
I20260808 08:29:07.461516 23471 external_mini_cluster.cc:1428] Setting key fc51b2844ee7b1c340e1bd09fc684253
I20260808 08:29:07.463630 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.235.195:0
--local_ip_for_outbound_sockets=127.22.235.195
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:33665
--builtin_ntp_servers=127.22.235.212:35321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 08:29:07.470314 24098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:07.462155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.194:33443 (local address 127.22.235.254:33665)
0808 08:29:07.462299 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:07.462304 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:07.463411 (+ 1107us) server_negotiation.cc:407] Connection header received
0808 08:29:07.465145 (+ 1734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:07.465148 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:07.465195 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:07.465263 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:07.466215 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:07.466958 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:07.468483 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:07.468723 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:07.468809 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:07.469872 (+ 1063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:07.469876 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:07.469878 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:29:07.469889 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:07.469909 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:29:07.469916 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:07.469974 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:07.470112 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 08:29:07.470171 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 08:29:07.471200 24536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:07.462539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:33665 (local address 127.22.235.194:33443)
0808 08:29:07.463257 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:07.463310 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:07.464925 (+ 1615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:07.465447 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:07.465457 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:07.465584 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:29:07.466075 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:07.466088 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:07.467099 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:07.467102 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:07.468369 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:07.468382 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:07.468873 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:07.468878 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:07.469037 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:07.469629 (+ 592us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 08:29:07.469631 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:29:07.469669 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 08:29:07.469733 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:29:07.469984 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:29:07.469993 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:29:07.470000 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 08:29:07.470105 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 08:29:07.470373 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":118,"threads_started":1}
I20260808 08:29:07.472245 24532 heartbeater.cc:347] Connected to a master server at 127.22.235.254:33665
I20260808 08:29:07.472468 24532 heartbeater.cc:464] Registering TS with master...
I20260808 08:29:07.473002 24532 heartbeater.cc:511] Master 127.22.235.254:33665 requested a full tablet report, sending...
I20260808 08:29:07.474028 24038 ts_manager.cc:194] Registered new tserver with Master: abaaac2a861a43ea806ba70a05fe357e (127.22.235.194:44061)
I20260808 08:29:07.474777 24038 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.235.194:33443
W20260808 08:29:07.577723 24537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:29:07.577960 24537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:29:07.578003 24537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:29:07.581938 24537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:29:07.581991 24537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:29:07.582046 24537 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.235.195
I20260808 08:29:07.586711 24537 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:35321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.235.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.235.195
--webserver_port=0
--tserver_master_addrs=127.22.235.254:33665
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.235.195
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:29:07.587711 24537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:29:07.589221 24537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:29:07.598573 24546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:07.598690 24544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:07.598656 24537 server_base.cc:1034] running on GCE node
W20260808 08:29:07.598584 24543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:07.599260 24537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:29:07.599874 24537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:29:07.601060 24537 hybrid_clock.cc:648] HybridClock initialized: now 1786177747601030 us; error 37 us; skew 500 ppm
I20260808 08:29:07.602810 24537 webserver.cc:460] Webserver started at http://127.22.235.195:36893/ using document root <none> and password file <none>
I20260808 08:29:07.603353 24537 fs_manager.cc:359] Metadata directory not provided
I20260808 08:29:07.603423 24537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:29:07.603633 24537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:29:07.605703 24537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-2/data/instance:
uuid: "40f46621258c402cbc150e0abe7bcab1"
format_stamp: "Formatted at 2026-08-08 08:29:07 on dist-test-slave-f7th"
server_key: "488829f8513098d02a577b6269e3e5bd"
server_key_iv: "9c5c5f331d7c1bf6f18a6a6098def3cb"
server_key_version: "encryptionkey@0"
I20260808 08:29:07.606215 24537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance:
uuid: "40f46621258c402cbc150e0abe7bcab1"
format_stamp: "Formatted at 2026-08-08 08:29:07 on dist-test-slave-f7th"
server_key: "488829f8513098d02a577b6269e3e5bd"
server_key_iv: "9c5c5f331d7c1bf6f18a6a6098def3cb"
server_key_version: "encryptionkey@0"
I20260808 08:29:07.609696 24537 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260808 08:29:07.611968 24552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:07.613147 24537 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:29:07.613252 24537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-2/wal
uuid: "40f46621258c402cbc150e0abe7bcab1"
format_stamp: "Formatted at 2026-08-08 08:29:07 on dist-test-slave-f7th"
server_key: "488829f8513098d02a577b6269e3e5bd"
server_key_iv: "9c5c5f331d7c1bf6f18a6a6098def3cb"
server_key_version: "encryptionkey@0"
I20260808 08:29:07.613339 24537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:29:07.641732 24537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:29:07.642447 24537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:29:07.642623 24537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:29:07.643302 24537 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 08:29:07.644279 24537 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 08:29:07.644328 24537 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:07.644382 24537 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 08:29:07.644409 24537 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:07.653405 24537 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.195:36709
I20260808 08:29:07.653434 24738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.195:36709 every 8 connection(s)
I20260808 08:29:07.654316 24537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-2/data/info.pb
I20260808 08:29:07.661576 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 24537
I20260808 08:29:07.661674 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-2/wal/instance
I20260808 08:29:07.661991 23471 external_mini_cluster.cc:1428] Setting key 62a203d27b1ab2fa007d514843c9cf97
I20260808 08:29:07.663408 24098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:07.656125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:36497 (local address 127.22.235.254:33665)
0808 08:29:07.656293 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:07.656298 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:07.657358 (+ 1060us) server_negotiation.cc:407] Connection header received
0808 08:29:07.658783 (+ 1425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:07.658786 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:07.658837 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:07.658908 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:07.659764 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:07.660271 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:07.661424 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:07.661659 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:07.661757 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:07.662947 (+ 1190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:07.662951 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:07.662954 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:29:07.662969 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:07.662994 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:29:07.663004 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:07.663074 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:07.663210 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 08:29:07.663265 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 08:29:07.664297 24741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:07.656527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:33665 (local address 127.22.235.195:36497)
0808 08:29:07.657210 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:07.657251 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:07.658594 (+ 1343us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:07.659064 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:07.659070 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:07.659151 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:29:07.659654 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:07.659668 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:07.660359 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:07.660361 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:07.661300 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:07.661310 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:07.661852 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:07.661856 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:07.662072 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:07.662684 (+ 612us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 08:29:07.662686 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:29:07.662724 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 08:29:07.662787 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:29:07.663078 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:29:07.663086 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:29:07.663091 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 08:29:07.663202 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 08:29:07.663449 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":111,"threads_started":1}
I20260808 08:29:07.665345 24739 heartbeater.cc:347] Connected to a master server at 127.22.235.254:33665
I20260808 08:29:07.665565 24739 heartbeater.cc:464] Registering TS with master...
I20260808 08:29:07.666054 24739 heartbeater.cc:511] Master 127.22.235.254:33665 requested a full tablet report, sending...
I20260808 08:29:07.667019 24038 ts_manager.cc:194] Registered new tserver with Master: 40f46621258c402cbc150e0abe7bcab1 (127.22.235.195:36709)
I20260808 08:29:07.667539 24038 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.235.195:36497
I20260808 08:29:07.675618 23471 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 08:29:07.680673 24098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:07.677542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43364 (local address 127.22.235.254:33665)
0808 08:29:07.677647 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:07.677650 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:07.677799 (+ 149us) server_negotiation.cc:407] Connection header received
0808 08:29:07.677893 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:07.677895 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:07.677931 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:07.678005 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:07.678216 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:07.678690 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:07.679406 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:07.679598 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:07.679662 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:07.680345 (+ 683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:07.680348 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:07.680350 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:29:07.680359 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:07.680373 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:29:07.680379 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:07.680426 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:07.680493 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 08:29:07.680537 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 08:29:07.683255 24038 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43364:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 08:29:07.686261 24038 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 08:29:07.699899 24755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:07.694290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.194:44061 (local address 127.0.0.1:36356)
0808 08:29:07.695000 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:07.695048 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:07.695173 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:07.695822 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:07.695827 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:07.695846 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:29:07.695949 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:07.695953 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:07.697358 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:07.697361 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:07.698432 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:07.698443 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:07.699324 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:07.699328 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:07.699483 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:07.699534 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 08:29:07.699639 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 08:29:07.699754 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":20,"thread_start_us":51,"threads_started":1}
I20260808 08:29:07.699909 24757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:07.694470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36356 (local address 127.22.235.194:44061)
0808 08:29:07.695316 (+ 846us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:07.695329 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:07.695353 (+ 24us) server_negotiation.cc:407] Connection header received
0808 08:29:07.695433 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:07.695439 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:07.695651 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:07.696083 (+ 432us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:29:07.696218 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:07.697236 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:07.698585 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:07.699211 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:07.699324 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:07.699511 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:07.699677 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 08:29:07.699772 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":735,"thread_start_us":152,"threads_started":1}
I20260808 08:29:07.700640 24760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:07.694651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:38733 (local address 127.0.0.1:43728)
0808 08:29:07.695377 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:07.695389 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:07.695459 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:07.696208 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:07.696212 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:07.696230 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:29:07.696336 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:07.696342 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:07.697771 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:07.697775 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:07.698914 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:07.698924 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:07.699951 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:07.699955 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:07.700324 (+ 369us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:07.700351 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 08:29:07.700405 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 08:29:07.700480 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"mutex_wait_us":20,"thread_start_us":96,"threads_started":1}
I20260808 08:29:07.701359 24761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:07.694823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43728 (local address 127.22.235.193:38733)
0808 08:29:07.695822 (+ 999us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:07.695833 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:07.695852 (+ 19us) server_negotiation.cc:407] Connection header received
0808 08:29:07.695917 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:07.695921 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:07.696062 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:07.696298 (+ 236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:29:07.696413 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:07.697636 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:07.699065 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:07.699833 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:07.700923 (+ 1090us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:07.701073 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:07.701117 (+ 44us) server_negotiation.cc:299] Negotiation successful
0808 08:29:07.701204 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":904,"thread_start_us":136,"threads_started":1}
I20260808 08:29:07.701413 24763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:07.694970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.195:36709 (local address 127.0.0.1:33100)
0808 08:29:07.695640 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:07.695651 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:07.695719 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:07.696171 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:07.696173 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:07.696190 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:29:07.696442 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:07.696446 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:07.698597 (+ 2151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:07.698600 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:07.699727 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:07.699739 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:07.701053 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:07.701056 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:07.701141 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:07.701162 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 08:29:07.701229 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 08:29:07.701298 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"mutex_wait_us":29,"thread_start_us":96,"threads_started":1}
I20260808 08:29:07.701824 24765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:07.695371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33100 (local address 127.22.235.195:36709)
0808 08:29:07.695673 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:07.695685 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:07.695711 (+ 26us) server_negotiation.cc:407] Connection header received
0808 08:29:07.695825 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:07.695830 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:07.696027 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:07.696190 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:29:07.696546 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:07.698476 (+ 1930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:07.699870 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:07.700668 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:07.700802 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:07.700985 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:07.701574 (+ 589us) server_negotiation.cc:299] Negotiation successful
0808 08:29:07.701671 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":119,"threads_started":1}
I20260808 08:29:07.702939 24427 tablet_service.cc:1467] Processing CreateTablet for tablet 1857fb5bf3de4fd191175d3905173825 (DEFAULT_TABLE table=test-table [id=05b3b30cdb5642b2b9e6de8639f65b16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:29:07.703907 24427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1857fb5bf3de4fd191175d3905173825. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:29:07.704135 24221 tablet_service.cc:1467] Processing CreateTablet for tablet 1857fb5bf3de4fd191175d3905173825 (DEFAULT_TABLE table=test-table [id=05b3b30cdb5642b2b9e6de8639f65b16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:29:07.704535 24627 tablet_service.cc:1467] Processing CreateTablet for tablet 1857fb5bf3de4fd191175d3905173825 (DEFAULT_TABLE table=test-table [id=05b3b30cdb5642b2b9e6de8639f65b16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 08:29:07.705224 24221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1857fb5bf3de4fd191175d3905173825. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:29:07.705524 24627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1857fb5bf3de4fd191175d3905173825. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:29:07.710238 24767 tablet_bootstrap.cc:492] T 1857fb5bf3de4fd191175d3905173825 P abaaac2a861a43ea806ba70a05fe357e: Bootstrap starting.
I20260808 08:29:07.712137 24768 tablet_bootstrap.cc:492] T 1857fb5bf3de4fd191175d3905173825 P dd5cc9b2cda24b3f96795f300c42b43e: Bootstrap starting.
I20260808 08:29:07.712430 24767 tablet_bootstrap.cc:654] T 1857fb5bf3de4fd191175d3905173825 P abaaac2a861a43ea806ba70a05fe357e: Neither blocks nor log segments found. Creating new log.
I20260808 08:29:07.712934 24770 tablet_bootstrap.cc:492] T 1857fb5bf3de4fd191175d3905173825 P 40f46621258c402cbc150e0abe7bcab1: Bootstrap starting.
I20260808 08:29:07.713443 24767 log.cc:824] T 1857fb5bf3de4fd191175d3905173825 P abaaac2a861a43ea806ba70a05fe357e: Log is configured to *not* fsync() on all Append() calls
I20260808 08:29:07.714215 24768 tablet_bootstrap.cc:654] T 1857fb5bf3de4fd191175d3905173825 P dd5cc9b2cda24b3f96795f300c42b43e: Neither blocks nor log segments found. Creating new log.
I20260808 08:29:07.714759 24770 tablet_bootstrap.cc:654] T 1857fb5bf3de4fd191175d3905173825 P 40f46621258c402cbc150e0abe7bcab1: Neither blocks nor log segments found. Creating new log.
I20260808 08:29:07.715008 24768 log.cc:824] T 1857fb5bf3de4fd191175d3905173825 P dd5cc9b2cda24b3f96795f300c42b43e: Log is configured to *not* fsync() on all Append() calls
I20260808 08:29:07.715593 24767 tablet_bootstrap.cc:492] T 1857fb5bf3de4fd191175d3905173825 P abaaac2a861a43ea806ba70a05fe357e: No bootstrap required, opened a new log
I20260808 08:29:07.715786 24767 ts_tablet_manager.cc:1397] T 1857fb5bf3de4fd191175d3905173825 P abaaac2a861a43ea806ba70a05fe357e: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20260808 08:29:07.715862 24770 log.cc:824] T 1857fb5bf3de4fd191175d3905173825 P 40f46621258c402cbc150e0abe7bcab1: Log is configured to *not* fsync() on all Append() calls
I20260808 08:29:07.716955 24768 tablet_bootstrap.cc:492] T 1857fb5bf3de4fd191175d3905173825 P dd5cc9b2cda24b3f96795f300c42b43e: No bootstrap required, opened a new log
I20260808 08:29:07.717139 24768 ts_tablet_manager.cc:1397] T 1857fb5bf3de4fd191175d3905173825 P dd5cc9b2cda24b3f96795f300c42b43e: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260808 08:29:07.717871 24770 tablet_bootstrap.cc:492] T 1857fb5bf3de4fd191175d3905173825 P 40f46621258c402cbc150e0abe7bcab1: No bootstrap required, opened a new log
I20260808 08:29:07.718067 24770 ts_tablet_manager.cc:1397] T 1857fb5bf3de4fd191175d3905173825 P 40f46621258c402cbc150e0abe7bcab1: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 08:29:07.719162 24767 raft_consensus.cc:348] T 1857fb5bf3de4fd191175d3905173825 P abaaac2a861a43ea806ba70a05fe357e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abaaac2a861a43ea806ba70a05fe357e" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44061 } } peers { permanent_uuid: "40f46621258c402cbc150e0abe7bcab1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 36709 } } peers { permanent_uuid: "dd5cc9b2cda24b3f96795f300c42b43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 38733 } }
I20260808 08:29:07.719362 24767 raft_consensus.cc:374] T 1857fb5bf3de4fd191175d3905173825 P abaaac2a861a43ea806ba70a05fe357e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:29:07.719414 24767 raft_consensus.cc:729] T 1857fb5bf3de4fd191175d3905173825 P abaaac2a861a43ea806ba70a05fe357e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abaaac2a861a43ea806ba70a05fe357e, State: Initialized, Role: FOLLOWER
I20260808 08:29:07.719799 24767 consensus_queue.cc:260] T 1857fb5bf3de4fd191175d3905173825 P abaaac2a861a43ea806ba70a05fe357e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abaaac2a861a43ea806ba70a05fe357e" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44061 } } peers { permanent_uuid: "40f46621258c402cbc150e0abe7bcab1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 36709 } } peers { permanent_uuid: "dd5cc9b2cda24b3f96795f300c42b43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 38733 } }
I20260808 08:29:07.720804 24767 ts_tablet_manager.cc:1428] T 1857fb5bf3de4fd191175d3905173825 P abaaac2a861a43ea806ba70a05fe357e: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260808 08:29:07.721148 24532 heartbeater.cc:503] Master 127.22.235.254:33665 was elected leader, sending a full tablet report...
I20260808 08:29:07.721235 24768 raft_consensus.cc:348] T 1857fb5bf3de4fd191175d3905173825 P dd5cc9b2cda24b3f96795f300c42b43e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abaaac2a861a43ea806ba70a05fe357e" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44061 } } peers { permanent_uuid: "40f46621258c402cbc150e0abe7bcab1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 36709 } } peers { permanent_uuid: "dd5cc9b2cda24b3f96795f300c42b43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 38733 } }
I20260808 08:29:07.721500 24768 raft_consensus.cc:374] T 1857fb5bf3de4fd191175d3905173825 P dd5cc9b2cda24b3f96795f300c42b43e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:29:07.721578 24768 raft_consensus.cc:729] T 1857fb5bf3de4fd191175d3905173825 P dd5cc9b2cda24b3f96795f300c42b43e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd5cc9b2cda24b3f96795f300c42b43e, State: Initialized, Role: FOLLOWER
I20260808 08:29:07.722003 24768 consensus_queue.cc:260] T 1857fb5bf3de4fd191175d3905173825 P dd5cc9b2cda24b3f96795f300c42b43e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abaaac2a861a43ea806ba70a05fe357e" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44061 } } peers { permanent_uuid: "40f46621258c402cbc150e0abe7bcab1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 36709 } } peers { permanent_uuid: "dd5cc9b2cda24b3f96795f300c42b43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 38733 } }
I20260808 08:29:07.722087 24770 raft_consensus.cc:348] T 1857fb5bf3de4fd191175d3905173825 P 40f46621258c402cbc150e0abe7bcab1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abaaac2a861a43ea806ba70a05fe357e" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44061 } } peers { permanent_uuid: "40f46621258c402cbc150e0abe7bcab1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 36709 } } peers { permanent_uuid: "dd5cc9b2cda24b3f96795f300c42b43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 38733 } }
I20260808 08:29:07.722316 24770 raft_consensus.cc:374] T 1857fb5bf3de4fd191175d3905173825 P 40f46621258c402cbc150e0abe7bcab1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:29:07.722375 24770 raft_consensus.cc:729] T 1857fb5bf3de4fd191175d3905173825 P 40f46621258c402cbc150e0abe7bcab1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40f46621258c402cbc150e0abe7bcab1, State: Initialized, Role: FOLLOWER
I20260808 08:29:07.722757 24770 consensus_queue.cc:260] T 1857fb5bf3de4fd191175d3905173825 P 40f46621258c402cbc150e0abe7bcab1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abaaac2a861a43ea806ba70a05fe357e" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44061 } } peers { permanent_uuid: "40f46621258c402cbc150e0abe7bcab1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 36709 } } peers { permanent_uuid: "dd5cc9b2cda24b3f96795f300c42b43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 38733 } }
I20260808 08:29:07.722975 24768 ts_tablet_manager.cc:1428] T 1857fb5bf3de4fd191175d3905173825 P dd5cc9b2cda24b3f96795f300c42b43e: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 08:29:07.723017 24325 heartbeater.cc:503] Master 127.22.235.254:33665 was elected leader, sending a full tablet report...
I20260808 08:29:07.724133 24739 heartbeater.cc:503] Master 127.22.235.254:33665 was elected leader, sending a full tablet report...
I20260808 08:29:07.724475 24770 ts_tablet_manager.cc:1428] T 1857fb5bf3de4fd191175d3905173825 P 40f46621258c402cbc150e0abe7bcab1: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260808 08:29:07.771869 24326 tablet.cc:2369] T 1857fb5bf3de4fd191175d3905173825 P dd5cc9b2cda24b3f96795f300c42b43e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 08:29:07.905268 24740 tablet.cc:2369] T 1857fb5bf3de4fd191175d3905173825 P 40f46621258c402cbc150e0abe7bcab1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 08:29:07.961423 24533 tablet.cc:2369] T 1857fb5bf3de4fd191175d3905173825 P abaaac2a861a43ea806ba70a05fe357e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 08:29:07.969534 24777 raft_consensus.cc:482] T 1857fb5bf3de4fd191175d3905173825 P 40f46621258c402cbc150e0abe7bcab1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 08:29:07.969741 24777 raft_consensus.cc:504] T 1857fb5bf3de4fd191175d3905173825 P 40f46621258c402cbc150e0abe7bcab1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abaaac2a861a43ea806ba70a05fe357e" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44061 } } peers { permanent_uuid: "40f46621258c402cbc150e0abe7bcab1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 36709 } } peers { permanent_uuid: "dd5cc9b2cda24b3f96795f300c42b43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 38733 } }
I20260808 08:29:07.970966 24777 leader_election.cc:288] T 1857fb5bf3de4fd191175d3905173825 P 40f46621258c402cbc150e0abe7bcab1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abaaac2a861a43ea806ba70a05fe357e (127.22.235.194:44061), dd5cc9b2cda24b3f96795f300c42b43e (127.22.235.193:38733)
I20260808 08:29:07.975780 24761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:07.971326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:37985 (local address 127.22.235.193:38733)
0808 08:29:07.971445 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:07.971450 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:07.971465 (+ 15us) server_negotiation.cc:407] Connection header received
0808 08:29:07.971615 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:07.971618 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:07.971663 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:07.971725 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:29:07.972227 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:07.973039 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:07.974584 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:07.975172 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:07.975289 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:07.975340 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:07.975575 (+ 235us) server_negotiation.cc:299] Negotiation successful
0808 08:29:07.975648 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 08:29:07.975780 24757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:07.971336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.235.195:49493 (local address 127.22.235.194:44061)
0808 08:29:07.971452 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:07.971457 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:07.971811 (+ 354us) server_negotiation.cc:407] Connection header received
0808 08:29:07.972015 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:07.972018 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:07.972066 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:07.972123 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 08:29:07.972369 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:07.973097 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:07.974583 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:07.975172 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:07.975289 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:07.975340 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:07.975575 (+ 235us) server_negotiation.cc:299] Negotiation successful
0808 08:29:07.975648 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 08:29:07.975814 24741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:07.971217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:38733 (local address 127.22.235.195:37985)
0808 08:29:07.971357 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:07.971373 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:07.971501 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:07.971748 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:07.971752 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:07.971768 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:29:07.971883 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:07.971889 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:07.973133 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:07.973135 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:07.974465 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:07.974476 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:07.975277 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:07.975281 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:07.975459 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:07.975478 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 08:29:07.975550 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 08:29:07.975657 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":112}
I20260808 08:29:07.975821 24778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:07.971217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.194:44061 (local address 127.22.235.195:49493)
0808 08:29:07.971726 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:07.971738 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:07.971887 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:07.972138 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:07.972140 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:07.972154 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 08:29:07.972262 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:07.972266 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:07.973188 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:07.973191 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:07.974465 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:07.974476 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:07.975277 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:07.975281 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:07.975441 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:07.975473 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 08:29:07.975559 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 08:29:07.975657 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"mutex_wait_us":273,"thread_start_us":134,"threads_started":1}
I20260808 08:29:07.976469 24455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1857fb5bf3de4fd191175d3905173825" candidate_uuid: "40f46621258c402cbc150e0abe7bcab1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abaaac2a861a43ea806ba70a05fe357e" is_pre_election: true
I20260808 08:29:07.976482 24251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1857fb5bf3de4fd191175d3905173825" candidate_uuid: "40f46621258c402cbc150e0abe7bcab1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd5cc9b2cda24b3f96795f300c42b43e" is_pre_election: true
I20260808 08:29:07.976773 24455 raft_consensus.cc:2454] T 1857fb5bf3de4fd191175d3905173825 P abaaac2a861a43ea806ba70a05fe357e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40f46621258c402cbc150e0abe7bcab1 in term 0.
I20260808 08:29:07.976799 24251 raft_consensus.cc:2454] T 1857fb5bf3de4fd191175d3905173825 P dd5cc9b2cda24b3f96795f300c42b43e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40f46621258c402cbc150e0abe7bcab1 in term 0.
I20260808 08:29:07.977244 24557 leader_election.cc:302] T 1857fb5bf3de4fd191175d3905173825 P 40f46621258c402cbc150e0abe7bcab1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 40f46621258c402cbc150e0abe7bcab1, abaaac2a861a43ea806ba70a05fe357e, dd5cc9b2cda24b3f96795f300c42b43e; no voters:
I20260808 08:29:07.977496 24777 raft_consensus.cc:2781] T 1857fb5bf3de4fd191175d3905173825 P 40f46621258c402cbc150e0abe7bcab1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 08:29:07.977578 24777 raft_consensus.cc:482] T 1857fb5bf3de4fd191175d3905173825 P 40f46621258c402cbc150e0abe7bcab1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 08:29:07.977612 24777 raft_consensus.cc:3037] T 1857fb5bf3de4fd191175d3905173825 P 40f46621258c402cbc150e0abe7bcab1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:29:07.978778 24777 raft_consensus.cc:504] T 1857fb5bf3de4fd191175d3905173825 P 40f46621258c402cbc150e0abe7bcab1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abaaac2a861a43ea806ba70a05fe357e" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44061 } } peers { permanent_uuid: "40f46621258c402cbc150e0abe7bcab1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 36709 } } peers { permanent_uuid: "dd5cc9b2cda24b3f96795f300c42b43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 38733 } }
I20260808 08:29:07.979117 24777 leader_election.cc:288] T 1857fb5bf3de4fd191175d3905173825 P 40f46621258c402cbc150e0abe7bcab1 [CANDIDATE]: Term 1 election: Requested vote from peers abaaac2a861a43ea806ba70a05fe357e (127.22.235.194:44061), dd5cc9b2cda24b3f96795f300c42b43e (127.22.235.193:38733)
I20260808 08:29:07.979444 24455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1857fb5bf3de4fd191175d3905173825" candidate_uuid: "40f46621258c402cbc150e0abe7bcab1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abaaac2a861a43ea806ba70a05fe357e"
I20260808 08:29:07.979444 24251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1857fb5bf3de4fd191175d3905173825" candidate_uuid: "40f46621258c402cbc150e0abe7bcab1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd5cc9b2cda24b3f96795f300c42b43e"
I20260808 08:29:07.979574 24251 raft_consensus.cc:3037] T 1857fb5bf3de4fd191175d3905173825 P dd5cc9b2cda24b3f96795f300c42b43e [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:29:07.979574 24455 raft_consensus.cc:3037] T 1857fb5bf3de4fd191175d3905173825 P abaaac2a861a43ea806ba70a05fe357e [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:29:07.980834 24251 raft_consensus.cc:2454] T 1857fb5bf3de4fd191175d3905173825 P dd5cc9b2cda24b3f96795f300c42b43e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40f46621258c402cbc150e0abe7bcab1 in term 1.
I20260808 08:29:07.980834 24455 raft_consensus.cc:2454] T 1857fb5bf3de4fd191175d3905173825 P abaaac2a861a43ea806ba70a05fe357e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40f46621258c402cbc150e0abe7bcab1 in term 1.
I20260808 08:29:07.981124 24557 leader_election.cc:302] T 1857fb5bf3de4fd191175d3905173825 P 40f46621258c402cbc150e0abe7bcab1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40f46621258c402cbc150e0abe7bcab1, abaaac2a861a43ea806ba70a05fe357e; no voters:
I20260808 08:29:07.981290 24777 raft_consensus.cc:2781] T 1857fb5bf3de4fd191175d3905173825 P 40f46621258c402cbc150e0abe7bcab1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:29:07.981485 24777 raft_consensus.cc:686] T 1857fb5bf3de4fd191175d3905173825 P 40f46621258c402cbc150e0abe7bcab1 [term 1 LEADER]: Becoming Leader. State: Replica: 40f46621258c402cbc150e0abe7bcab1, State: Running, Role: LEADER
I20260808 08:29:07.981802 24777 consensus_queue.cc:237] T 1857fb5bf3de4fd191175d3905173825 P 40f46621258c402cbc150e0abe7bcab1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abaaac2a861a43ea806ba70a05fe357e" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44061 } } peers { permanent_uuid: "40f46621258c402cbc150e0abe7bcab1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 36709 } } peers { permanent_uuid: "dd5cc9b2cda24b3f96795f300c42b43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 38733 } }
I20260808 08:29:07.984808 24036 catalog_manager.cc:5495] T 1857fb5bf3de4fd191175d3905173825 P 40f46621258c402cbc150e0abe7bcab1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 40f46621258c402cbc150e0abe7bcab1 (127.22.235.195). New cstate: current_term: 1 leader_uuid: "40f46621258c402cbc150e0abe7bcab1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abaaac2a861a43ea806ba70a05fe357e" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40f46621258c402cbc150e0abe7bcab1" member_type: VOTER last_known_addr { host: "127.22.235.195" port: 36709 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd5cc9b2cda24b3f96795f300c42b43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 38733 } health_report { overall_health: UNKNOWN } } }
I20260808 08:29:08.059121 24765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:08.056084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33112 (local address 127.22.235.195:36709)
0808 08:29:08.056209 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:08.056212 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:08.056241 (+ 29us) server_negotiation.cc:407] Connection header received
0808 08:29:08.056318 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:08.056320 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:08.056374 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:08.056451 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:29:08.056658 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:08.057155 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:08.057916 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:08.058124 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:08.058188 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:08.058406 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:29:08.058802 (+ 396us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 08:29:08.058890 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:08.058920 (+ 30us) server_negotiation.cc:299] Negotiation successful
0808 08:29:08.058977 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 08:29:08.065025 24251 raft_consensus.cc:1261] T 1857fb5bf3de4fd191175d3905173825 P dd5cc9b2cda24b3f96795f300c42b43e [term 1 FOLLOWER]: Refusing update from remote peer 40f46621258c402cbc150e0abe7bcab1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:29:08.065029 24455 raft_consensus.cc:1261] T 1857fb5bf3de4fd191175d3905173825 P abaaac2a861a43ea806ba70a05fe357e [term 1 FOLLOWER]: Refusing update from remote peer 40f46621258c402cbc150e0abe7bcab1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:29:08.065712 24777 consensus_queue.cc:1035] T 1857fb5bf3de4fd191175d3905173825 P 40f46621258c402cbc150e0abe7bcab1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abaaac2a861a43ea806ba70a05fe357e" member_type: VOTER last_known_addr { host: "127.22.235.194" port: 44061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:29:08.065914 24779 consensus_queue.cc:1035] T 1857fb5bf3de4fd191175d3905173825 P 40f46621258c402cbc150e0abe7bcab1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd5cc9b2cda24b3f96795f300c42b43e" member_type: VOTER last_known_addr { host: "127.22.235.193" port: 38733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:29:08.073755 24788 mvcc.cc:205] Tried to move back new op lower bound from 7316184056066842624 to 7316184055735738368. Current Snapshot: MvccSnapshot[applied={T|T < 7316184056066842624}]
I20260808 08:29:08.082870 24036 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:29:08.083051 24036 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 08:29:08.085534 24036 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7f89304a85b4432d909a236f5874d142: Sending DeleteTablet for 3 replicas of tablet 1857fb5bf3de4fd191175d3905173825
I20260808 08:29:08.086279 24221 tablet_service.cc:1514] Processing DeleteTablet for tablet 1857fb5bf3de4fd191175d3905173825 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:29:08 UTC) from {username='slave'} at 127.0.0.1:43728
I20260808 08:29:08.086336 24427 tablet_service.cc:1514] Processing DeleteTablet for tablet 1857fb5bf3de4fd191175d3905173825 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:29:08 UTC) from {username='slave'} at 127.0.0.1:36356
I20260808 08:29:08.086319 24627 tablet_service.cc:1514] Processing DeleteTablet for tablet 1857fb5bf3de4fd191175d3905173825 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 08:29:08 UTC) from {username='slave'} at 127.0.0.1:33100
I20260808 08:29:08.086760 24798 tablet_replica.cc:330] T 1857fb5bf3de4fd191175d3905173825 P dd5cc9b2cda24b3f96795f300c42b43e: stopping tablet replica
I20260808 08:29:08.086937 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 24131
I20260808 08:29:08.087091 24798 raft_consensus.cc:2229] T 1857fb5bf3de4fd191175d3905173825 P dd5cc9b2cda24b3f96795f300c42b43e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 08:29:08.096864 24801 tablet_replica.cc:330] T 1857fb5bf3de4fd191175d3905173825 P 40f46621258c402cbc150e0abe7bcab1: stopping tablet replica
I20260808 08:29:08.097364 24801 raft_consensus.cc:2229] T 1857fb5bf3de4fd191175d3905173825 P 40f46621258c402cbc150e0abe7bcab1 [term 1 LEADER]: Raft consensus shutting down.
I20260808 08:29:08.097908 24801 raft_consensus.cc:2258] T 1857fb5bf3de4fd191175d3905173825 P 40f46621258c402cbc150e0abe7bcab1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:29:08.098469 24802 tablet_replica.cc:330] T 1857fb5bf3de4fd191175d3905173825 P abaaac2a861a43ea806ba70a05fe357e: stopping tablet replica
I20260808 08:29:08.098848 24802 raft_consensus.cc:2229] T 1857fb5bf3de4fd191175d3905173825 P abaaac2a861a43ea806ba70a05fe357e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 08:29:08.099251 24802 raft_consensus.cc:2258] T 1857fb5bf3de4fd191175d3905173825 P abaaac2a861a43ea806ba70a05fe357e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 08:29:08.122310 24802 ts_tablet_manager.cc:1905] T 1857fb5bf3de4fd191175d3905173825 P abaaac2a861a43ea806ba70a05fe357e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:29:08.124221 24801 ts_tablet_manager.cc:1905] T 1857fb5bf3de4fd191175d3905173825 P 40f46621258c402cbc150e0abe7bcab1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 08:29:08.125391 24802 ts_tablet_manager.cc:1918] T 1857fb5bf3de4fd191175d3905173825 P abaaac2a861a43ea806ba70a05fe357e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 08:29:08.125504 24802 log.cc:1196] T 1857fb5bf3de4fd191175d3905173825 P abaaac2a861a43ea806ba70a05fe357e: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-1/wal/wals/1857fb5bf3de4fd191175d3905173825
I20260808 08:29:08.125823 24802 ts_tablet_manager.cc:1939] T 1857fb5bf3de4fd191175d3905173825 P abaaac2a861a43ea806ba70a05fe357e: Deleting consensus metadata
I20260808 08:29:08.127213 24801 ts_tablet_manager.cc:1918] T 1857fb5bf3de4fd191175d3905173825 P 40f46621258c402cbc150e0abe7bcab1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 08:29:08.127245 24023 catalog_manager.cc:4841] TS abaaac2a861a43ea806ba70a05fe357e (127.22.235.194:44061): tablet 1857fb5bf3de4fd191175d3905173825 (table test-table [id=05b3b30cdb5642b2b9e6de8639f65b16]) successfully deleted
I20260808 08:29:08.127439 24801 log.cc:1196] T 1857fb5bf3de4fd191175d3905173825 P 40f46621258c402cbc150e0abe7bcab1: Deleting WAL directory at /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786177493204855-23471-0/minicluster-data/ts-2/wal/wals/1857fb5bf3de4fd191175d3905173825
I20260808 08:29:08.127774 24801 ts_tablet_manager.cc:1939] T 1857fb5bf3de4fd191175d3905173825 P 40f46621258c402cbc150e0abe7bcab1: Deleting consensus metadata
I20260808 08:29:08.128897 24023 catalog_manager.cc:4841] TS 40f46621258c402cbc150e0abe7bcab1 (127.22.235.195:36709): tablet 1857fb5bf3de4fd191175d3905173825 (table test-table [id=05b3b30cdb5642b2b9e6de8639f65b16]) successfully deleted
W20260808 08:29:08.130115 24022 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.235.193:38733 (error 108)
I20260808 08:29:08.130946 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 24328
I20260808 08:29:08.131211 24763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:08.130886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.193:38733 (local address 127.0.0.1:43744)
0808 08:29:08.131054 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:08.131127 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.193:38733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 08:29:08.140242 24022 catalog_manager.cc:4584] TS dd5cc9b2cda24b3f96795f300c42b43e (127.22.235.193:38733): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1857fb5bf3de4fd191175d3905173825: Network error: Client connection negotiation failed: client connection to 127.22.235.193:38733: connect: Connection refused (error 111)
I20260808 08:29:08.146946 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 24537
I20260808 08:29:08.158272 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 23996
2026-08-08T08:29:08Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1357 ms)
[----------] 6 tests from EncryptionPolicyTest (11566 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786177493204855-23471-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 08:29:08 dist-test-slave-f7th krb5kdc[24807](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 08:29:08 dist-test-slave-f7th krb5kdc[24807](info): set up 2 sockets
Aug 08 08:29:08 dist-test-slave-f7th krb5kdc[24807](info): commencing operation
krb5kdc: starting...
W20260808 08:29:09.328445 23471 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Aug 08 08:29:09 dist-test-slave-f7th krb5kdc[24807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177749, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T08:29:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:29:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786177493204855-23471-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786177493204855-23471-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 08:29:09.469779 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41675
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786177493204855-23471-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41675
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:29:09.582931 24830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:29:09.583242 24830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:29:09.583302 24830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:29:09.587293 24830 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:29:09.587352 24830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:29:09.587378 24830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:29:09.587401 24830 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:29:09.587430 24830 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:29:09.592600 24830 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41675
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786177493204855-23471-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24830
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:29:09.593623 24830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:29:09.594616 24830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:29:09.603044 24842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:09.603057 24840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:09.603260 24830 server_base.cc:1034] running on GCE node
W20260808 08:29:09.603053 24839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:09.603829 24830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:29:09.604980 24830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:29:09.606165 24830 hybrid_clock.cc:648] HybridClock initialized: now 1786177749606150 us; error 26 us; skew 500 ppm
Aug 08 08:29:09 dist-test-slave-f7th krb5kdc[24807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177749, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:29:09.608857 24830 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 08:29:09.609724 24830 webserver.cc:460] Webserver started at http://127.0.0.1:41491/ using document root <none> and password file <none>
I20260808 08:29:09.610250 24830 fs_manager.cc:359] Metadata directory not provided
I20260808 08:29:09.610316 24830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:29:09.610512 24830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:29:09.612504 24830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "9821a9a26c2d42caaff39c90a9150c80"
format_stamp: "Formatted at 2026-08-08 08:29:09 on dist-test-slave-f7th"
server_key: "9b4892c6898c4aadfc6cc375ca2376bd"
server_key_iv: "fcf4989b3b90887f8a511e5d960c0b88"
server_key_version: "encryptionkey@0"
I20260808 08:29:09.613071 24830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "9821a9a26c2d42caaff39c90a9150c80"
format_stamp: "Formatted at 2026-08-08 08:29:09 on dist-test-slave-f7th"
server_key: "9b4892c6898c4aadfc6cc375ca2376bd"
server_key_iv: "fcf4989b3b90887f8a511e5d960c0b88"
server_key_version: "encryptionkey@0"
I20260808 08:29:09.616546 24830 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 08:29:09.618956 24851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:09.620185 24830 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:29:09.620297 24830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "9821a9a26c2d42caaff39c90a9150c80"
format_stamp: "Formatted at 2026-08-08 08:29:09 on dist-test-slave-f7th"
server_key: "9b4892c6898c4aadfc6cc375ca2376bd"
server_key_iv: "fcf4989b3b90887f8a511e5d960c0b88"
server_key_version: "encryptionkey@0"
I20260808 08:29:09.620383 24830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:29:09.642659 24830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:29:09.645363 24830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:29:09.645555 24830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:29:09.652452 24830 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41675
I20260808 08:29:09.652474 24931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41675 every 8 connection(s)
I20260808 08:29:09.653437 24830 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:29:09.655799 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 24830
I20260808 08:29:09.655936 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:29:09.656278 23471 external_mini_cluster.cc:1428] Setting key b162b8eca3a66087d646e95fe0095c97
I20260808 08:29:09.656509 24932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:29:09.663620 24932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9821a9a26c2d42caaff39c90a9150c80: Bootstrap starting.
Aug 08 08:29:09 dist-test-slave-f7th krb5kdc[24807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177749, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 08:29:09.666072 24932 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9821a9a26c2d42caaff39c90a9150c80: Neither blocks nor log segments found. Creating new log.
I20260808 08:29:09.666950 24932 log.cc:824] T 00000000000000000000000000000000 P 9821a9a26c2d42caaff39c90a9150c80: Log is configured to *not* fsync() on all Append() calls
I20260808 08:29:09.669039 24932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9821a9a26c2d42caaff39c90a9150c80: No bootstrap required, opened a new log
I20260808 08:29:09.671244 24940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:09.657696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41537 (local address 127.0.0.1:41675)
0808 08:29:09.658232 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:09.658251 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:09.658291 (+ 40us) server_negotiation.cc:407] Connection header received
0808 08:29:09.659273 (+ 982us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:09.659292 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:09.659630 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:09.659905 (+ 275us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:09.660886 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:09.661892 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:09.663054 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:09.663748 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:09.663912 (+ 164us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:09.666667 (+ 2755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:09.666672 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:09.666685 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:09.666713 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:09.668349 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:09.668951 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:09.668956 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:09.668961 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:09.669068 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:09.669425 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:09.669428 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:09.669429 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:09.669797 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:09.669992 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:09.670350 (+ 358us) server_negotiation.cc:299] Negotiation successful
0808 08:29:09.670562 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":135,"threads_started":1}
I20260808 08:29:09.672573 24932 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9821a9a26c2d42caaff39c90a9150c80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9821a9a26c2d42caaff39c90a9150c80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41675 } }
I20260808 08:29:09.672829 24932 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9821a9a26c2d42caaff39c90a9150c80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:29:09.672892 24932 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9821a9a26c2d42caaff39c90a9150c80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9821a9a26c2d42caaff39c90a9150c80, State: Initialized, Role: FOLLOWER
I20260808 08:29:09.673358 24932 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9821a9a26c2d42caaff39c90a9150c80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9821a9a26c2d42caaff39c90a9150c80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41675 } }
I20260808 08:29:09.673497 24932 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9821a9a26c2d42caaff39c90a9150c80 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:29:09.673561 24932 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9821a9a26c2d42caaff39c90a9150c80 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:29:09.673655 24932 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9821a9a26c2d42caaff39c90a9150c80 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:29:09.674077 23471 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 08:29:09.674599 24932 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9821a9a26c2d42caaff39c90a9150c80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9821a9a26c2d42caaff39c90a9150c80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41675 } }
I20260808 08:29:09.674918 24932 leader_election.cc:302] T 00000000000000000000000000000000 P 9821a9a26c2d42caaff39c90a9150c80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9821a9a26c2d42caaff39c90a9150c80; no voters:
I20260808 08:29:09.675210 24932 leader_election.cc:288] T 00000000000000000000000000000000 P 9821a9a26c2d42caaff39c90a9150c80 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:29:09.675336 24945 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9821a9a26c2d42caaff39c90a9150c80 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:29:09.675549 24945 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9821a9a26c2d42caaff39c90a9150c80 [term 1 LEADER]: Becoming Leader. State: Replica: 9821a9a26c2d42caaff39c90a9150c80, State: Running, Role: LEADER
I20260808 08:29:09.676425 24945 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9821a9a26c2d42caaff39c90a9150c80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9821a9a26c2d42caaff39c90a9150c80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41675 } }
I20260808 08:29:09.676506 24932 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9821a9a26c2d42caaff39c90a9150c80 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:29:09.678133 24951 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9821a9a26c2d42caaff39c90a9150c80 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9821a9a26c2d42caaff39c90a9150c80. Latest consensus state: current_term: 1 leader_uuid: "9821a9a26c2d42caaff39c90a9150c80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9821a9a26c2d42caaff39c90a9150c80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41675 } } }
I20260808 08:29:09.678253 24951 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9821a9a26c2d42caaff39c90a9150c80 [sys.catalog]: This master's current role is: LEADER
I20260808 08:29:09.678568 24952 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9821a9a26c2d42caaff39c90a9150c80 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9821a9a26c2d42caaff39c90a9150c80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9821a9a26c2d42caaff39c90a9150c80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41675 } } }
I20260808 08:29:09.678858 24952 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9821a9a26c2d42caaff39c90a9150c80 [sys.catalog]: This master's current role is: LEADER
I20260808 08:29:09.679733 24978 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:29:09.682809 24978 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:29:09.684931 24940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:09.676267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38183 (local address 127.0.0.1:41675)
0808 08:29:09.676382 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:09.676386 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:09.676400 (+ 14us) server_negotiation.cc:407] Connection header received
0808 08:29:09.676468 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:09.676471 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:09.676513 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:09.676601 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:09.677528 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:09.678327 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:09.679567 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:09.679796 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:09.679898 (+ 102us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:09.681398 (+ 1500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:09.681403 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:09.681405 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:09.681431 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:09.683027 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:09.683578 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:09.683581 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:09.683583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:09.683675 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:09.684030 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:09.684034 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:09.684036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:09.684251 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:09.684402 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:09.684696 (+ 294us) server_negotiation.cc:299] Negotiation successful
0808 08:29:09.684783 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 08:29:09.688704 24978 catalog_manager.cc:1347] Generated new cluster ID: f0986b7a12004cff815986ea278c9daf
I20260808 08:29:09.688761 24978 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:29:09.710225 24978 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:29:09.711107 24978 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:29:09.718420 24978 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9821a9a26c2d42caaff39c90a9150c80: Generated new TSK 0
I20260808 08:29:09.719055 24978 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 08:29:09.729403 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 24830
W20260808 08:29:09.749392 24805 connection.cc:535] client connection to 127.0.0.1:41675 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41675): Connection reset by peer (error 104)
2026-08-08T08:29:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1575 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786177493204855-23471-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 08:29:09 dist-test-slave-f7th krb5kdc[24991](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 08:29:09 dist-test-slave-f7th krb5kdc[24991](info): set up 2 sockets
Aug 08 08:29:09 dist-test-slave-f7th krb5kdc[24991](info): commencing operation
krb5kdc: starting...
W20260808 08:29:10.937716 23471 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Aug 08 08:29:11 dist-test-slave-f7th krb5kdc[24991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177751, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T08:29:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:29:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786177493204855-23471-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786177493204855-23471-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 08:29:11.055191 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45501
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786177493204855-23471-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45501
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:29:11.164906 25012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:29:11.165155 25012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:29:11.165212 25012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:29:11.169184 25012 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:29:11.169236 25012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:29:11.169263 25012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:29:11.169284 25012 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:29:11.169313 25012 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:29:11.174335 25012 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45501
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786177493204855-23471-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:29:11.175273 25012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:29:11.176231 25012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:29:11.184273 25022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:11.184294 25024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:11.184422 25012 server_base.cc:1034] running on GCE node
W20260808 08:29:11.184497 25027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:11.185091 25012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:29:11.186154 25012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:29:11.187315 25012 hybrid_clock.cc:648] HybridClock initialized: now 1786177751187297 us; error 31 us; skew 500 ppm
Aug 08 08:29:11 dist-test-slave-f7th krb5kdc[24991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177751, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:29:11.189949 25012 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 08:29:11.190810 25012 webserver.cc:460] Webserver started at http://127.0.0.1:35357/ using document root <none> and password file <none>
I20260808 08:29:11.191344 25012 fs_manager.cc:359] Metadata directory not provided
I20260808 08:29:11.191408 25012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:29:11.191596 25012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:29:11.193624 25012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "15ec107022ba4c278feb5726000e8053"
format_stamp: "Formatted at 2026-08-08 08:29:11 on dist-test-slave-f7th"
server_key: "4ac1fc6fcb6d1a62984f5882d5a0382f"
server_key_iv: "a2522b7d61b4cc296f2581bc7ec403aa"
server_key_version: "encryptionkey@0"
I20260808 08:29:11.194150 25012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "15ec107022ba4c278feb5726000e8053"
format_stamp: "Formatted at 2026-08-08 08:29:11 on dist-test-slave-f7th"
server_key: "4ac1fc6fcb6d1a62984f5882d5a0382f"
server_key_iv: "a2522b7d61b4cc296f2581bc7ec403aa"
server_key_version: "encryptionkey@0"
I20260808 08:29:11.197685 25012 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.001s
I20260808 08:29:11.200014 25038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:11.201203 25012 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:29:11.201320 25012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "15ec107022ba4c278feb5726000e8053"
format_stamp: "Formatted at 2026-08-08 08:29:11 on dist-test-slave-f7th"
server_key: "4ac1fc6fcb6d1a62984f5882d5a0382f"
server_key_iv: "a2522b7d61b4cc296f2581bc7ec403aa"
server_key_version: "encryptionkey@0"
I20260808 08:29:11.201411 25012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:29:11.220757 25012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:29:11.223331 25012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:29:11.223500 25012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:29:11.230093 25012 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45501
I20260808 08:29:11.230108 25121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45501 every 8 connection(s)
I20260808 08:29:11.231050 25012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:29:11.233767 25123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:29:11.239069 25123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15ec107022ba4c278feb5726000e8053: Bootstrap starting.
I20260808 08:29:11.241075 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 25012
I20260808 08:29:11.241199 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:29:11.241413 25123 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 15ec107022ba4c278feb5726000e8053: Neither blocks nor log segments found. Creating new log.
I20260808 08:29:11.241459 23471 external_mini_cluster.cc:1428] Setting key 60ebd645e1473048b26572a8ff8a1205
I20260808 08:29:11.242144 25123 log.cc:824] T 00000000000000000000000000000000 P 15ec107022ba4c278feb5726000e8053: Log is configured to *not* fsync() on all Append() calls
I20260808 08:29:11.244398 25123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15ec107022ba4c278feb5726000e8053: No bootstrap required, opened a new log
I20260808 08:29:11.248885 25123 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15ec107022ba4c278feb5726000e8053 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15ec107022ba4c278feb5726000e8053" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45501 } }
I20260808 08:29:11.249146 25123 raft_consensus.cc:374] T 00000000000000000000000000000000 P 15ec107022ba4c278feb5726000e8053 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:29:11.249233 25123 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15ec107022ba4c278feb5726000e8053 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15ec107022ba4c278feb5726000e8053, State: Initialized, Role: FOLLOWER
I20260808 08:29:11.249773 25123 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15ec107022ba4c278feb5726000e8053 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15ec107022ba4c278feb5726000e8053" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45501 } }
I20260808 08:29:11.250029 25123 raft_consensus.cc:388] T 00000000000000000000000000000000 P 15ec107022ba4c278feb5726000e8053 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:29:11.250114 25123 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15ec107022ba4c278feb5726000e8053 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:29:11.250213 25123 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15ec107022ba4c278feb5726000e8053 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:29:11.251379 25123 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15ec107022ba4c278feb5726000e8053 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15ec107022ba4c278feb5726000e8053" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45501 } }
I20260808 08:29:11.251767 25123 leader_election.cc:302] T 00000000000000000000000000000000 P 15ec107022ba4c278feb5726000e8053 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 15ec107022ba4c278feb5726000e8053; no voters:
I20260808 08:29:11.252161 25123 leader_election.cc:288] T 00000000000000000000000000000000 P 15ec107022ba4c278feb5726000e8053 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:29:11.252260 25129 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15ec107022ba4c278feb5726000e8053 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:29:11.252547 25128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:11.242527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42679 (local address 127.0.0.1:45501)
0808 08:29:11.242945 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:11.242963 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:11.243012 (+ 49us) server_negotiation.cc:407] Connection header received
0808 08:29:11.243812 (+ 800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:11.243835 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:11.244252 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:11.244593 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:11.245054 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:11.246489 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:11.247894 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:11.248603 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:11.248779 (+ 176us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:11.249988 (+ 1209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:11.249995 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:11.250013 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:29:11.250528 (+ 515us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:11.250599 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:29:11.250632 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:11.250767 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:11.251413 (+ 646us) server_negotiation.cc:299] Negotiation successful
0808 08:29:11.251558 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":132,"threads_started":1}
I20260808 08:29:11.252553 25129 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15ec107022ba4c278feb5726000e8053 [term 1 LEADER]: Becoming Leader. State: Replica: 15ec107022ba4c278feb5726000e8053, State: Running, Role: LEADER
I20260808 08:29:11.253142 25129 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15ec107022ba4c278feb5726000e8053 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15ec107022ba4c278feb5726000e8053" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45501 } }
I20260808 08:29:11.253539 25123 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15ec107022ba4c278feb5726000e8053 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:29:11.255043 25130 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15ec107022ba4c278feb5726000e8053 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15ec107022ba4c278feb5726000e8053. Latest consensus state: current_term: 1 leader_uuid: "15ec107022ba4c278feb5726000e8053" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15ec107022ba4c278feb5726000e8053" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45501 } } }
I20260808 08:29:11.255257 25130 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15ec107022ba4c278feb5726000e8053 [sys.catalog]: This master's current role is: LEADER
I20260808 08:29:11.255450 25131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15ec107022ba4c278feb5726000e8053 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "15ec107022ba4c278feb5726000e8053" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15ec107022ba4c278feb5726000e8053" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45501 } } }
I20260808 08:29:11.255923 25131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15ec107022ba4c278feb5726000e8053 [sys.catalog]: This master's current role is: LEADER
I20260808 08:29:11.256400 23471 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260808 08:29:11.258682 25150 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 15ec107022ba4c278feb5726000e8053: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 08:29:11.258755 25150 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 08:29:11.258841 25148 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:29:11.259822 25148 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:29:11.262756 25128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:11.258646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47165 (local address 127.0.0.1:45501)
0808 08:29:11.258751 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:11.258755 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:11.259301 (+ 546us) server_negotiation.cc:407] Connection header received
0808 08:29:11.259396 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:11.259398 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:11.259441 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:11.259505 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:11.259776 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:11.260341 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:11.261221 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:11.261465 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:11.261554 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:11.262351 (+ 797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:11.262354 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:11.262356 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:29:11.262368 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:11.262393 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:29:11.262403 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:11.262485 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:11.262593 (+ 108us) server_negotiation.cc:299] Negotiation successful
0808 08:29:11.262625 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 08:29:11.265936 25148 catalog_manager.cc:1347] Generated new cluster ID: 5dfabd13a88e42fab69df82ffbb073da
I20260808 08:29:11.265995 25148 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:29:11.277024 25148 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:29:11.277887 25148 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:29:11.283353 25148 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 15ec107022ba4c278feb5726000e8053: Generated new TSK 0
I20260808 08:29:11.284014 25148 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 08:29:11.297147 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 25012
W20260808 08:29:11.311210 24986 connection.cc:535] client connection to 127.0.0.1:45501 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45501): Connection reset by peer (error 104)
2026-08-08T08:29:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1561 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786177493204855-23471-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 08:29:11 dist-test-slave-f7th krb5kdc[25164](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 08:29:11 dist-test-slave-f7th krb5kdc[25164](info): set up 2 sockets
Aug 08 08:29:11 dist-test-slave-f7th krb5kdc[25164](info): commencing operation
krb5kdc: starting...
W20260808 08:29:12.467393 23471 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Aug 08 08:29:12 dist-test-slave-f7th krb5kdc[25164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T08:29:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:29:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786177493204855-23471-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786177493204855-23471-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 08:29:12.589578 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41059
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786177493204855-23471-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41059
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:29:12.702025 25185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:29:12.702287 25185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:29:12.702348 25185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:29:12.706367 25185 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:29:12.706426 25185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:29:12.706452 25185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:29:12.706472 25185 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:29:12.706492 25185 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:29:12.712010 25185 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41059
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786177493204855-23471-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41059
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25185
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:29:12.713052 25185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:29:12.714025 25185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:29:12.722062 25192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:12.722114 25185 server_base.cc:1034] running on GCE node
W20260808 08:29:12.722061 25195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:12.722121 25193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:12.722886 25185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:29:12.723917 25185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:29:12.725071 25185 hybrid_clock.cc:648] HybridClock initialized: now 1786177752725056 us; error 30 us; skew 500 ppm
Aug 08 08:29:12 dist-test-slave-f7th krb5kdc[25164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177752, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:29:12.732281 25185 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 08:29:12.733199 25185 webserver.cc:460] Webserver started at http://127.0.0.1:33331/ using document root <none> and password file <none>
I20260808 08:29:12.733748 25185 fs_manager.cc:359] Metadata directory not provided
I20260808 08:29:12.733815 25185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:29:12.734020 25185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:29:12.736019 25185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "4b00ec4888604148be99067e40237e3a"
format_stamp: "Formatted at 2026-08-08 08:29:12 on dist-test-slave-f7th"
server_key: "7130253f0b227c55505104743b6c3ecd"
server_key_iv: "a29b5c9a60525f3000b9575315ab06fc"
server_key_version: "encryptionkey@0"
I20260808 08:29:12.736553 25185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "4b00ec4888604148be99067e40237e3a"
format_stamp: "Formatted at 2026-08-08 08:29:12 on dist-test-slave-f7th"
server_key: "7130253f0b227c55505104743b6c3ecd"
server_key_iv: "a29b5c9a60525f3000b9575315ab06fc"
server_key_version: "encryptionkey@0"
I20260808 08:29:12.740100 25185 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 08:29:12.742420 25208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:12.743536 25185 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:29:12.743664 25185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "4b00ec4888604148be99067e40237e3a"
format_stamp: "Formatted at 2026-08-08 08:29:12 on dist-test-slave-f7th"
server_key: "7130253f0b227c55505104743b6c3ecd"
server_key_iv: "a29b5c9a60525f3000b9575315ab06fc"
server_key_version: "encryptionkey@0"
I20260808 08:29:12.743752 25185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:29:12.748122 25185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:29:12.751019 25185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:29:12.751183 25185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:29:12.757653 25185 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41059
I20260808 08:29:12.757660 25296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41059 every 8 connection(s)
I20260808 08:29:12.758535 25185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:29:12.761204 25297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:29:12.765340 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 25185
I20260808 08:29:12.765484 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:29:12.765760 23471 external_mini_cluster.cc:1428] Setting key 5b1a0f152108567f7a7b2e5e114614e7
I20260808 08:29:12.766957 25297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b00ec4888604148be99067e40237e3a: Bootstrap starting.
I20260808 08:29:12.769307 25297 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b00ec4888604148be99067e40237e3a: Neither blocks nor log segments found. Creating new log.
I20260808 08:29:12.770138 25297 log.cc:824] T 00000000000000000000000000000000 P 4b00ec4888604148be99067e40237e3a: Log is configured to *not* fsync() on all Append() calls
I20260808 08:29:12.770937 25302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:12.767231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56233 (local address 127.0.0.1:41059)
0808 08:29:12.767672 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:12.767687 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:12.767722 (+ 35us) server_negotiation.cc:407] Connection header received
0808 08:29:12.768350 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:12.768368 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:12.768683 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:12.768996 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:12.769795 (+ 799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:12.769800 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:12.769812 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:29:12.769823 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:12.769876 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:29:12.769895 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:12.769997 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:12.770116 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 08:29:12.770274 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":127,"threads_started":1}
I20260808 08:29:12.772022 25297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b00ec4888604148be99067e40237e3a: No bootstrap required, opened a new log
I20260808 08:29:12.773813 23471 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 08:29:12.775954 25297 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b00ec4888604148be99067e40237e3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b00ec4888604148be99067e40237e3a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41059 } }
I20260808 08:29:12.776168 25297 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b00ec4888604148be99067e40237e3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:29:12.776222 25297 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b00ec4888604148be99067e40237e3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b00ec4888604148be99067e40237e3a, State: Initialized, Role: FOLLOWER
I20260808 08:29:12.776710 25297 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b00ec4888604148be99067e40237e3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b00ec4888604148be99067e40237e3a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41059 } }
I20260808 08:29:12.776875 25297 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b00ec4888604148be99067e40237e3a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:29:12.776945 25297 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b00ec4888604148be99067e40237e3a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:29:12.777055 25297 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b00ec4888604148be99067e40237e3a [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:29:12.777246 25302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:12.775685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51069 (local address 127.0.0.1:41059)
0808 08:29:12.775784 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:12.775788 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:12.775976 (+ 188us) server_negotiation.cc:407] Connection header received
0808 08:29:12.776102 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:12.776105 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:12.776135 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:12.776197 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:12.776893 (+ 696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:12.776897 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:12.776899 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:29:12.776911 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:12.776939 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:29:12.776951 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:12.777027 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:12.777095 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 08:29:12.777133 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 08:29:12.778149 25297 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b00ec4888604148be99067e40237e3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b00ec4888604148be99067e40237e3a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41059 } }
I20260808 08:29:12.778482 25297 leader_election.cc:302] T 00000000000000000000000000000000 P 4b00ec4888604148be99067e40237e3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b00ec4888604148be99067e40237e3a; no voters:
I20260808 08:29:12.778761 25297 leader_election.cc:288] T 00000000000000000000000000000000 P 4b00ec4888604148be99067e40237e3a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:29:12.778855 25310 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b00ec4888604148be99067e40237e3a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:29:12.779070 25310 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b00ec4888604148be99067e40237e3a [term 1 LEADER]: Becoming Leader. State: Replica: 4b00ec4888604148be99067e40237e3a, State: Running, Role: LEADER
I20260808 08:29:12.779371 25310 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b00ec4888604148be99067e40237e3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b00ec4888604148be99067e40237e3a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41059 } }
I20260808 08:29:12.779939 25297 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b00ec4888604148be99067e40237e3a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:29:12.780961 25311 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b00ec4888604148be99067e40237e3a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b00ec4888604148be99067e40237e3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b00ec4888604148be99067e40237e3a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41059 } } }
I20260808 08:29:12.780990 25312 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b00ec4888604148be99067e40237e3a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b00ec4888604148be99067e40237e3a. Latest consensus state: current_term: 1 leader_uuid: "4b00ec4888604148be99067e40237e3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b00ec4888604148be99067e40237e3a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41059 } } }
I20260808 08:29:12.781168 25312 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b00ec4888604148be99067e40237e3a [sys.catalog]: This master's current role is: LEADER
I20260808 08:29:12.781172 25311 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b00ec4888604148be99067e40237e3a [sys.catalog]: This master's current role is: LEADER
I20260808 08:29:12.782076 25325 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:29:12.785102 25325 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:29:12.790486 25325 catalog_manager.cc:1347] Generated new cluster ID: 2b2904f90acd44b7b3ab1c569a95963f
I20260808 08:29:12.790548 25325 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:29:12.803063 25325 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:29:12.803361 25325 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:29:12.810272 25325 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b00ec4888604148be99067e40237e3a: Generated new TSK 0
I20260808 08:29:12.810920 25325 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 08:29:12.819502 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 25185
2026-08-08T08:29:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1525 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786177493204855-23471-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 08:29:12 dist-test-slave-f7th krb5kdc[25344](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 08:29:12 dist-test-slave-f7th krb5kdc[25344](info): set up 2 sockets
Aug 08 08:29:12 dist-test-slave-f7th krb5kdc[25344](info): commencing operation
krb5kdc: starting...
W20260808 08:29:13.998922 23471 mini_kdc.cc:121] Time spent starting KDC: real 1.148s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Aug 08 08:29:14 dist-test-slave-f7th krb5kdc[25344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T08:29:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:29:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786177493204855-23471-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786177493204855-23471-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 08:29:14.115629 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39253
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786177493204855-23471-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39253
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:29:14.225143 25363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:29:14.225401 25363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:29:14.225440 25363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:29:14.229317 25363 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:29:14.229367 25363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:29:14.229384 25363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:29:14.229405 25363 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:29:14.229424 25363 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:29:14.234477 25363 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39253
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786177493204855-23471-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39253
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:29:14.235467 25363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:29:14.236415 25363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 08:29:14.244715 25363 server_base.cc:1034] running on GCE node
W20260808 08:29:14.244659 25374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:14.244733 25372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:14.244668 25376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:14.245347 25363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:29:14.246423 25363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:29:14.247591 25363 hybrid_clock.cc:648] HybridClock initialized: now 1786177754247576 us; error 28 us; skew 500 ppm
Aug 08 08:29:14 dist-test-slave-f7th krb5kdc[25344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177754, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:29:14.250231 25363 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 08:29:14.251161 25363 webserver.cc:460] Webserver started at http://127.0.0.1:33115/ using document root <none> and password file <none>
I20260808 08:29:14.251720 25363 fs_manager.cc:359] Metadata directory not provided
I20260808 08:29:14.251791 25363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:29:14.252007 25363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:29:14.254048 25363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "1d2d6d630ad54b7aab3ca7c5717e338b"
format_stamp: "Formatted at 2026-08-08 08:29:14 on dist-test-slave-f7th"
server_key: "98fb5685d2e10422ef6d0ac5a7cade8c"
server_key_iv: "814fd7958dd5c87cc8c5aea20e378022"
server_key_version: "encryptionkey@0"
I20260808 08:29:14.254576 25363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "1d2d6d630ad54b7aab3ca7c5717e338b"
format_stamp: "Formatted at 2026-08-08 08:29:14 on dist-test-slave-f7th"
server_key: "98fb5685d2e10422ef6d0ac5a7cade8c"
server_key_iv: "814fd7958dd5c87cc8c5aea20e378022"
server_key_version: "encryptionkey@0"
I20260808 08:29:14.258225 25363 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 08:29:14.260568 25387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:14.261796 25363 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:29:14.261925 25363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "1d2d6d630ad54b7aab3ca7c5717e338b"
format_stamp: "Formatted at 2026-08-08 08:29:14 on dist-test-slave-f7th"
server_key: "98fb5685d2e10422ef6d0ac5a7cade8c"
server_key_iv: "814fd7958dd5c87cc8c5aea20e378022"
server_key_version: "encryptionkey@0"
I20260808 08:29:14.262033 25363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:29:14.279227 25363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:29:14.281901 25363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:29:14.282080 25363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:29:14.289011 25363 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39253
I20260808 08:29:14.289047 25475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39253 every 8 connection(s)
I20260808 08:29:14.289927 25363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:29:14.291133 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 25363
I20260808 08:29:14.291276 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:29:14.291602 23471 external_mini_cluster.cc:1428] Setting key b2d17caff8cb2e08c54720ef8de0f4a6
I20260808 08:29:14.293161 25476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:29:14.298671 25476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d2d6d630ad54b7aab3ca7c5717e338b: Bootstrap starting.
Aug 08 08:29:14 dist-test-slave-f7th krb5kdc[25344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 08:29:14.301159 25476 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d2d6d630ad54b7aab3ca7c5717e338b: Neither blocks nor log segments found. Creating new log.
I20260808 08:29:14.301951 25476 log.cc:824] T 00000000000000000000000000000000 P 1d2d6d630ad54b7aab3ca7c5717e338b: Log is configured to *not* fsync() on all Append() calls
I20260808 08:29:14.303733 25476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d2d6d630ad54b7aab3ca7c5717e338b: No bootstrap required, opened a new log
I20260808 08:29:14.304863 25479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:14.293044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46711 (local address 127.0.0.1:39253)
0808 08:29:14.293474 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:14.293496 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:14.293531 (+ 35us) server_negotiation.cc:407] Connection header received
0808 08:29:14.294147 (+ 616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:14.294162 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:14.294480 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:14.294795 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:14.295574 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:14.296565 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:14.297409 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:14.297974 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:14.298117 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:14.300673 (+ 2556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:14.300678 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:14.300691 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:14.300717 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:14.302296 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:14.302692 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:14.302697 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:14.302702 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:14.302808 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:14.303058 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:14.303061 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:14.303063 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:14.303432 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:14.303619 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:14.303885 (+ 266us) server_negotiation.cc:299] Negotiation successful
0808 08:29:14.304155 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":145,"threads_started":1}
I20260808 08:29:14.306806 25476 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d2d6d630ad54b7aab3ca7c5717e338b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d2d6d630ad54b7aab3ca7c5717e338b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39253 } }
I20260808 08:29:14.307035 25476 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d2d6d630ad54b7aab3ca7c5717e338b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:29:14.307094 25476 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d2d6d630ad54b7aab3ca7c5717e338b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d2d6d630ad54b7aab3ca7c5717e338b, State: Initialized, Role: FOLLOWER
I20260808 08:29:14.307519 25476 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d2d6d630ad54b7aab3ca7c5717e338b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d2d6d630ad54b7aab3ca7c5717e338b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39253 } }
I20260808 08:29:14.307675 25476 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d2d6d630ad54b7aab3ca7c5717e338b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:29:14.307745 25476 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d2d6d630ad54b7aab3ca7c5717e338b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:29:14.307812 23471 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 08:29:14.307886 25476 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d2d6d630ad54b7aab3ca7c5717e338b [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:29:14.308744 25476 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d2d6d630ad54b7aab3ca7c5717e338b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d2d6d630ad54b7aab3ca7c5717e338b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39253 } }
I20260808 08:29:14.309088 25476 leader_election.cc:302] T 00000000000000000000000000000000 P 1d2d6d630ad54b7aab3ca7c5717e338b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d2d6d630ad54b7aab3ca7c5717e338b; no voters:
I20260808 08:29:14.309388 25476 leader_election.cc:288] T 00000000000000000000000000000000 P 1d2d6d630ad54b7aab3ca7c5717e338b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:29:14.309947 25495 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d2d6d630ad54b7aab3ca7c5717e338b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:29:14.310155 25495 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d2d6d630ad54b7aab3ca7c5717e338b [term 1 LEADER]: Becoming Leader. State: Replica: 1d2d6d630ad54b7aab3ca7c5717e338b, State: Running, Role: LEADER
I20260808 08:29:14.310508 25495 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d2d6d630ad54b7aab3ca7c5717e338b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d2d6d630ad54b7aab3ca7c5717e338b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39253 } }
I20260808 08:29:14.310720 25476 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d2d6d630ad54b7aab3ca7c5717e338b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:29:14.312191 25496 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d2d6d630ad54b7aab3ca7c5717e338b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1d2d6d630ad54b7aab3ca7c5717e338b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d2d6d630ad54b7aab3ca7c5717e338b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39253 } } }
I20260808 08:29:14.312386 25496 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d2d6d630ad54b7aab3ca7c5717e338b [sys.catalog]: This master's current role is: LEADER
I20260808 08:29:14.312764 25498 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d2d6d630ad54b7aab3ca7c5717e338b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d2d6d630ad54b7aab3ca7c5717e338b. Latest consensus state: current_term: 1 leader_uuid: "1d2d6d630ad54b7aab3ca7c5717e338b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d2d6d630ad54b7aab3ca7c5717e338b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39253 } } }
I20260808 08:29:14.313719 25498 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d2d6d630ad54b7aab3ca7c5717e338b [sys.catalog]: This master's current role is: LEADER
W20260808 08:29:14.315690 25516 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1d2d6d630ad54b7aab3ca7c5717e338b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 08:29:14.315757 25516 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 08:29:14.315825 25515 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:29:14.316787 25515 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:29:14.318372 25479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:14.309803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43843 (local address 127.0.0.1:39253)
0808 08:29:14.309888 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:14.309892 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:14.310273 (+ 381us) server_negotiation.cc:407] Connection header received
0808 08:29:14.310429 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:14.310432 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:14.310473 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:14.310535 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:14.311226 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:14.311958 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:14.313123 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:14.313431 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:14.313540 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:14.315086 (+ 1546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:14.315090 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:14.315092 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:14.315115 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:14.316591 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:14.317118 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:14.317121 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:14.317123 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:14.317210 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:14.317506 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:14.317509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:14.317511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:14.317713 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:14.317835 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:14.318109 (+ 274us) server_negotiation.cc:299] Negotiation successful
0808 08:29:14.318224 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 08:29:14.322528 25515 catalog_manager.cc:1347] Generated new cluster ID: 3333bbdd2f884e7c983d9cecb6d3e6ab
I20260808 08:29:14.322595 25515 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:29:14.328603 25515 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:29:14.329802 25515 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:29:14.336081 25515 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1d2d6d630ad54b7aab3ca7c5717e338b: Generated new TSK 0
I20260808 08:29:14.336925 25515 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 08:29:14.342567 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 25363
2026-08-08T08:29:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1514 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786177493204855-23471-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 08:29:14 dist-test-slave-f7th krb5kdc[25529](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 08:29:14 dist-test-slave-f7th krb5kdc[25529](info): set up 2 sockets
Aug 08 08:29:14 dist-test-slave-f7th krb5kdc[25529](info): commencing operation
krb5kdc: starting...
W20260808 08:29:15.503882 23471 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Aug 08 08:29:15 dist-test-slave-f7th krb5kdc[25529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177755, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T08:29:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:29:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786177493204855-23471-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786177493204855-23471-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 08:29:15.621670 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35485
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786177493204855-23471-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35485
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:29:15.731241 25555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:29:15.731508 25555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:29:15.731570 25555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:29:15.735482 25555 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:29:15.735538 25555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:29:15.735565 25555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:29:15.735587 25555 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:29:15.735616 25555 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:29:15.740687 25555 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35485
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786177493204855-23471-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35485
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25555
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:29:15.741641 25555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:29:15.742614 25555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:29:15.750622 25566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:15.750636 25568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:15.750633 25565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:15.750801 25555 server_base.cc:1034] running on GCE node
I20260808 08:29:15.751449 25555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:29:15.752494 25555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:29:15.753656 25555 hybrid_clock.cc:648] HybridClock initialized: now 1786177755753641 us; error 30 us; skew 500 ppm
Aug 08 08:29:15 dist-test-slave-f7th krb5kdc[25529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177755, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:29:15.756186 25555 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 08:29:15.757078 25555 webserver.cc:460] Webserver started at http://127.0.0.1:37849/ using document root <none> and password file <none>
I20260808 08:29:15.757591 25555 fs_manager.cc:359] Metadata directory not provided
I20260808 08:29:15.757645 25555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:29:15.757815 25555 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:29:15.759738 25555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "8a0b911ff8474afdb261211b36c9c29c"
format_stamp: "Formatted at 2026-08-08 08:29:15 on dist-test-slave-f7th"
server_key: "b6951b5cac5e8f1fce159cbf35108254"
server_key_iv: "6f4d5cc63455568b0e1fae3e1289223c"
server_key_version: "encryptionkey@0"
I20260808 08:29:15.760224 25555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "8a0b911ff8474afdb261211b36c9c29c"
format_stamp: "Formatted at 2026-08-08 08:29:15 on dist-test-slave-f7th"
server_key: "b6951b5cac5e8f1fce159cbf35108254"
server_key_iv: "6f4d5cc63455568b0e1fae3e1289223c"
server_key_version: "encryptionkey@0"
I20260808 08:29:15.763767 25555 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 08:29:15.766186 25578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:15.767359 25555 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 08:29:15.767486 25555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "8a0b911ff8474afdb261211b36c9c29c"
format_stamp: "Formatted at 2026-08-08 08:29:15 on dist-test-slave-f7th"
server_key: "b6951b5cac5e8f1fce159cbf35108254"
server_key_iv: "6f4d5cc63455568b0e1fae3e1289223c"
server_key_version: "encryptionkey@0"
I20260808 08:29:15.767586 25555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:29:15.780928 25555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:29:15.783607 25555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:29:15.783777 25555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:29:15.790774 25555 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35485
I20260808 08:29:15.790783 25668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35485 every 8 connection(s)
I20260808 08:29:15.791733 25555 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:29:15.794536 25672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:29:15.797235 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 25555
I20260808 08:29:15.797369 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:29:15.797679 23471 external_mini_cluster.cc:1428] Setting key 9cbf31768674a535e43fb6951f3aa87e
I20260808 08:29:15.800726 25672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a0b911ff8474afdb261211b36c9c29c: Bootstrap starting.
I20260808 08:29:15.802872 25672 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a0b911ff8474afdb261211b36c9c29c: Neither blocks nor log segments found. Creating new log.
I20260808 08:29:15.803578 25672 log.cc:824] T 00000000000000000000000000000000 P 8a0b911ff8474afdb261211b36c9c29c: Log is configured to *not* fsync() on all Append() calls
I20260808 08:29:15.805469 25672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a0b911ff8474afdb261211b36c9c29c: No bootstrap required, opened a new log
I20260808 08:29:15.806398 25675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:15.799170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43927 (local address 127.0.0.1:35485)
0808 08:29:15.799618 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:15.799633 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:15.799669 (+ 36us) server_negotiation.cc:407] Connection header received
0808 08:29:15.800286 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:15.800303 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:15.800695 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:15.801030 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:15.801440 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:15.802398 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:15.803311 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:15.803813 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:15.803963 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:15.804785 (+ 822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:15.804790 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:15.804804 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:29:15.805177 (+ 373us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:15.805231 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:29:15.805254 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:15.805342 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:15.805499 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 08:29:15.805714 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":114,"threads_started":1}
I20260808 08:29:15.809455 23471 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 08:29:15.809518 25672 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a0b911ff8474afdb261211b36c9c29c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a0b911ff8474afdb261211b36c9c29c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35485 } }
I20260808 08:29:15.809772 25672 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a0b911ff8474afdb261211b36c9c29c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:29:15.809823 25672 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a0b911ff8474afdb261211b36c9c29c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a0b911ff8474afdb261211b36c9c29c, State: Initialized, Role: FOLLOWER
I20260808 08:29:15.810289 25672 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a0b911ff8474afdb261211b36c9c29c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a0b911ff8474afdb261211b36c9c29c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35485 } }
I20260808 08:29:15.810459 25672 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a0b911ff8474afdb261211b36c9c29c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:29:15.810527 25672 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a0b911ff8474afdb261211b36c9c29c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:29:15.810624 25672 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a0b911ff8474afdb261211b36c9c29c [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:29:15.811681 25672 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a0b911ff8474afdb261211b36c9c29c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a0b911ff8474afdb261211b36c9c29c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35485 } }
I20260808 08:29:15.812058 25672 leader_election.cc:302] T 00000000000000000000000000000000 P 8a0b911ff8474afdb261211b36c9c29c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a0b911ff8474afdb261211b36c9c29c; no voters:
I20260808 08:29:15.812405 25672 leader_election.cc:288] T 00000000000000000000000000000000 P 8a0b911ff8474afdb261211b36c9c29c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:29:15.812493 25683 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a0b911ff8474afdb261211b36c9c29c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:29:15.812703 25683 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a0b911ff8474afdb261211b36c9c29c [term 1 LEADER]: Becoming Leader. State: Replica: 8a0b911ff8474afdb261211b36c9c29c, State: Running, Role: LEADER
I20260808 08:29:15.813038 25683 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a0b911ff8474afdb261211b36c9c29c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a0b911ff8474afdb261211b36c9c29c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35485 } }
I20260808 08:29:15.813656 25672 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a0b911ff8474afdb261211b36c9c29c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:29:15.814180 25685 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a0b911ff8474afdb261211b36c9c29c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a0b911ff8474afdb261211b36c9c29c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a0b911ff8474afdb261211b36c9c29c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35485 } } }
I20260808 08:29:15.814376 25685 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a0b911ff8474afdb261211b36c9c29c [sys.catalog]: This master's current role is: LEADER
I20260808 08:29:15.814641 25685 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a0b911ff8474afdb261211b36c9c29c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a0b911ff8474afdb261211b36c9c29c. Latest consensus state: current_term: 1 leader_uuid: "8a0b911ff8474afdb261211b36c9c29c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a0b911ff8474afdb261211b36c9c29c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35485 } } }
I20260808 08:29:15.814781 25685 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a0b911ff8474afdb261211b36c9c29c [sys.catalog]: This master's current role is: LEADER
I20260808 08:29:15.815109 25693 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:29:15.816565 25675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:15.811762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51059 (local address 127.0.0.1:35485)
0808 08:29:15.811934 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:15.811937 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:15.812076 (+ 139us) server_negotiation.cc:407] Connection header received
0808 08:29:15.812208 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:15.812210 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:15.812252 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:15.812336 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:15.812695 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:15.813392 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:15.814521 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:15.814889 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:15.814964 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:15.816125 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:15.816128 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:15.816131 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:29:15.816144 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:15.816172 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:29:15.816186 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:15.816257 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:15.816382 (+ 125us) server_negotiation.cc:299] Negotiation successful
0808 08:29:15.816435 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260808 08:29:15.818755 25693 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:29:15.824462 25693 catalog_manager.cc:1347] Generated new cluster ID: 832c0ad85c24449f9fad995ffb55876a
I20260808 08:29:15.824520 25693 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:29:15.844660 25693 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:29:15.845638 25693 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:29:15.850754 25693 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a0b911ff8474afdb261211b36c9c29c: Generated new TSK 0
I20260808 08:29:15.851419 25693 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 08:29:15.865244 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 25555
2026-08-08T08:29:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1529 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786177493204855-23471-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 08:29:15 dist-test-slave-f7th krb5kdc[25723](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 08:29:15 dist-test-slave-f7th krb5kdc[25723](info): set up 2 sockets
Aug 08 08:29:15 dist-test-slave-f7th krb5kdc[25723](info): commencing operation
krb5kdc: starting...
W20260808 08:29:17.029488 23471 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Aug 08 08:29:17 dist-test-slave-f7th krb5kdc[25723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177757, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T08:29:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:29:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786177493204855-23471-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786177493204855-23471-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 08:29:17.149092 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35853
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786177493204855-23471-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35853
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:29:17.261895 25740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:29:17.262151 25740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:29:17.262200 25740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:29:17.266269 25740 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:29:17.266335 25740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:29:17.266362 25740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:29:17.266391 25740 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:29:17.266422 25740 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:29:17.271775 25740 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35853
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786177493204855-23471-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25740
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:29:17.272857 25740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:29:17.273856 25740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:29:17.282119 25748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:17.282176 25746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:17.282125 25751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:17.282362 25740 server_base.cc:1034] running on GCE node
I20260808 08:29:17.283064 25740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:29:17.284191 25740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:29:17.285341 25740 hybrid_clock.cc:648] HybridClock initialized: now 1786177757285314 us; error 43 us; skew 500 ppm
Aug 08 08:29:17 dist-test-slave-f7th krb5kdc[25723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177757, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:29:17.288026 25740 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 08:29:17.288937 25740 webserver.cc:460] Webserver started at http://127.0.0.1:46471/ using document root <none> and password file <none>
I20260808 08:29:17.289494 25740 fs_manager.cc:359] Metadata directory not provided
I20260808 08:29:17.289559 25740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:29:17.289762 25740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:29:17.291723 25740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "c82612a4c31c4de1aaa36dba6f5aa7f5"
format_stamp: "Formatted at 2026-08-08 08:29:17 on dist-test-slave-f7th"
server_key: "03fcd6ad978d7b5490494d34e3d413e1"
server_key_iv: "8842f00c637bd032bb4552ce74b8f582"
server_key_version: "encryptionkey@0"
I20260808 08:29:17.292207 25740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "c82612a4c31c4de1aaa36dba6f5aa7f5"
format_stamp: "Formatted at 2026-08-08 08:29:17 on dist-test-slave-f7th"
server_key: "03fcd6ad978d7b5490494d34e3d413e1"
server_key_iv: "8842f00c637bd032bb4552ce74b8f582"
server_key_version: "encryptionkey@0"
I20260808 08:29:17.295928 25740 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 08:29:17.298307 25769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:17.299507 25740 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 08:29:17.299618 25740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "c82612a4c31c4de1aaa36dba6f5aa7f5"
format_stamp: "Formatted at 2026-08-08 08:29:17 on dist-test-slave-f7th"
server_key: "03fcd6ad978d7b5490494d34e3d413e1"
server_key_iv: "8842f00c637bd032bb4552ce74b8f582"
server_key_version: "encryptionkey@0"
I20260808 08:29:17.299714 25740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:29:17.304240 25740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:29:17.307283 25740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:29:17.307447 25740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:29:17.314155 25740 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35853
I20260808 08:29:17.314190 25855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35853 every 8 connection(s)
I20260808 08:29:17.315133 25740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:29:17.317983 25857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:29:17.323423 25857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c82612a4c31c4de1aaa36dba6f5aa7f5: Bootstrap starting.
I20260808 08:29:17.325124 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 25740
I20260808 08:29:17.325248 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:29:17.325485 23471 external_mini_cluster.cc:1428] Setting key 29d6fc87bda7517eba63671ec9fe39cb
I20260808 08:29:17.326023 25857 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c82612a4c31c4de1aaa36dba6f5aa7f5: Neither blocks nor log segments found. Creating new log.
I20260808 08:29:17.326874 25857 log.cc:824] T 00000000000000000000000000000000 P c82612a4c31c4de1aaa36dba6f5aa7f5: Log is configured to *not* fsync() on all Append() calls
I20260808 08:29:17.329088 25857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c82612a4c31c4de1aaa36dba6f5aa7f5: No bootstrap required, opened a new log
I20260808 08:29:17.330610 25861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:17.326577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54499 (local address 127.0.0.1:35853)
0808 08:29:17.327070 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:17.327088 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:17.327132 (+ 44us) server_negotiation.cc:407] Connection header received
0808 08:29:17.327908 (+ 776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:17.327932 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:17.328269 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:17.328663 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:17.329296 (+ 633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:17.329301 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:17.329316 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:29:17.329327 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:17.329384 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:29:17.329411 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:17.329528 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:17.329593 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 08:29:17.329737 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":135,"threads_started":1}
I20260808 08:29:17.332968 25857 raft_consensus.cc:348] T 00000000000000000000000000000000 P c82612a4c31c4de1aaa36dba6f5aa7f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c82612a4c31c4de1aaa36dba6f5aa7f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35853 } }
I20260808 08:29:17.333205 25857 raft_consensus.cc:374] T 00000000000000000000000000000000 P c82612a4c31c4de1aaa36dba6f5aa7f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:29:17.333249 23471 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 08:29:17.333267 25857 raft_consensus.cc:729] T 00000000000000000000000000000000 P c82612a4c31c4de1aaa36dba6f5aa7f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c82612a4c31c4de1aaa36dba6f5aa7f5, State: Initialized, Role: FOLLOWER
I20260808 08:29:17.333650 25857 consensus_queue.cc:260] T 00000000000000000000000000000000 P c82612a4c31c4de1aaa36dba6f5aa7f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c82612a4c31c4de1aaa36dba6f5aa7f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35853 } }
I20260808 08:29:17.333767 25857 raft_consensus.cc:388] T 00000000000000000000000000000000 P c82612a4c31c4de1aaa36dba6f5aa7f5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:29:17.333828 25857 raft_consensus.cc:482] T 00000000000000000000000000000000 P c82612a4c31c4de1aaa36dba6f5aa7f5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:29:17.333928 25857 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c82612a4c31c4de1aaa36dba6f5aa7f5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:29:17.334841 25857 raft_consensus.cc:504] T 00000000000000000000000000000000 P c82612a4c31c4de1aaa36dba6f5aa7f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c82612a4c31c4de1aaa36dba6f5aa7f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35853 } }
I20260808 08:29:17.335193 25857 leader_election.cc:302] T 00000000000000000000000000000000 P c82612a4c31c4de1aaa36dba6f5aa7f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c82612a4c31c4de1aaa36dba6f5aa7f5; no voters:
I20260808 08:29:17.335525 25857 leader_election.cc:288] T 00000000000000000000000000000000 P c82612a4c31c4de1aaa36dba6f5aa7f5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:29:17.335616 25873 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c82612a4c31c4de1aaa36dba6f5aa7f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:29:17.335819 25873 raft_consensus.cc:686] T 00000000000000000000000000000000 P c82612a4c31c4de1aaa36dba6f5aa7f5 [term 1 LEADER]: Becoming Leader. State: Replica: c82612a4c31c4de1aaa36dba6f5aa7f5, State: Running, Role: LEADER
I20260808 08:29:17.336198 25873 consensus_queue.cc:237] T 00000000000000000000000000000000 P c82612a4c31c4de1aaa36dba6f5aa7f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c82612a4c31c4de1aaa36dba6f5aa7f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35853 } }
I20260808 08:29:17.336830 25857 sys_catalog.cc:564] T 00000000000000000000000000000000 P c82612a4c31c4de1aaa36dba6f5aa7f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:29:17.337175 25861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:17.335219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45631 (local address 127.0.0.1:35853)
0808 08:29:17.335318 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:17.335322 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:17.335607 (+ 285us) server_negotiation.cc:407] Connection header received
0808 08:29:17.335782 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:17.335785 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:17.335823 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:17.335904 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:17.336746 (+ 842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:17.336750 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:17.336752 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:29:17.336765 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:17.336791 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:29:17.336804 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:17.336858 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:17.337025 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 08:29:17.337065 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 08:29:17.337570 25873 sys_catalog.cc:455] T 00000000000000000000000000000000 P c82612a4c31c4de1aaa36dba6f5aa7f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c82612a4c31c4de1aaa36dba6f5aa7f5. Latest consensus state: current_term: 1 leader_uuid: "c82612a4c31c4de1aaa36dba6f5aa7f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c82612a4c31c4de1aaa36dba6f5aa7f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35853 } } }
I20260808 08:29:17.337754 25873 sys_catalog.cc:458] T 00000000000000000000000000000000 P c82612a4c31c4de1aaa36dba6f5aa7f5 [sys.catalog]: This master's current role is: LEADER
I20260808 08:29:17.337983 25877 sys_catalog.cc:455] T 00000000000000000000000000000000 P c82612a4c31c4de1aaa36dba6f5aa7f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c82612a4c31c4de1aaa36dba6f5aa7f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c82612a4c31c4de1aaa36dba6f5aa7f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35853 } } }
I20260808 08:29:17.338073 25877 sys_catalog.cc:458] T 00000000000000000000000000000000 P c82612a4c31c4de1aaa36dba6f5aa7f5 [sys.catalog]: This master's current role is: LEADER
I20260808 08:29:17.338382 25890 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:29:17.341642 25890 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:29:17.347210 25890 catalog_manager.cc:1347] Generated new cluster ID: 46c20d74cd9d44cf8ac2a2a9a73fbc03
I20260808 08:29:17.347282 25890 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:29:17.363792 25890 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:29:17.364135 25890 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:29:17.374435 25890 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c82612a4c31c4de1aaa36dba6f5aa7f5: Generated new TSK 0
I20260808 08:29:17.375211 25890 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 08:29:17.381510 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 25740
2026-08-08T08:29:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1513 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786177493204855-23471-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 08:29:17 dist-test-slave-f7th krb5kdc[25912](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 08:29:17 dist-test-slave-f7th krb5kdc[25912](info): set up 2 sockets
Aug 08 08:29:17 dist-test-slave-f7th krb5kdc[25912](info): commencing operation
krb5kdc: starting...
W20260808 08:29:18.542825 23471 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Aug 08 08:29:18 dist-test-slave-f7th krb5kdc[25912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177758, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T08:29:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:29:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786177493204855-23471-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786177493204855-23471-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 08:29:18.660286 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43243
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786177493204855-23471-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43243
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:29:18.771072 25933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:29:18.771327 25933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:29:18.771386 25933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:29:18.775303 25933 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:29:18.775363 25933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:29:18.775390 25933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:29:18.775413 25933 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:29:18.775449 25933 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:29:18.780494 25933 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43243
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786177493204855-23471-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:29:18.781500 25933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:29:18.782505 25933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:29:18.790657 25942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:18.790709 25947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:18.790729 25944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:18.790827 25933 server_base.cc:1034] running on GCE node
I20260808 08:29:18.791455 25933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:29:18.792467 25933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:29:18.793628 25933 hybrid_clock.cc:648] HybridClock initialized: now 1786177758793604 us; error 41 us; skew 500 ppm
Aug 08 08:29:18 dist-test-slave-f7th krb5kdc[25912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177758, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:29:18.799901 25933 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 08:29:18.800835 25933 webserver.cc:460] Webserver started at http://127.0.0.1:39413/ using document root <none> and password file <none>
I20260808 08:29:18.801378 25933 fs_manager.cc:359] Metadata directory not provided
I20260808 08:29:18.801436 25933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:29:18.801656 25933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:29:18.803682 25933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "f631027ed4f848a58b97cf02ccac9616"
format_stamp: "Formatted at 2026-08-08 08:29:18 on dist-test-slave-f7th"
server_key: "e4e742eca27839bca0d440c564e506dd"
server_key_iv: "e7d3d31b8b80f1548b0e259012587037"
server_key_version: "encryptionkey@0"
I20260808 08:29:18.804301 25933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "f631027ed4f848a58b97cf02ccac9616"
format_stamp: "Formatted at 2026-08-08 08:29:18 on dist-test-slave-f7th"
server_key: "e4e742eca27839bca0d440c564e506dd"
server_key_iv: "e7d3d31b8b80f1548b0e259012587037"
server_key_version: "encryptionkey@0"
I20260808 08:29:18.808048 25933 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 08:29:18.810499 25960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:18.811681 25933 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:29:18.811800 25933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "f631027ed4f848a58b97cf02ccac9616"
format_stamp: "Formatted at 2026-08-08 08:29:18 on dist-test-slave-f7th"
server_key: "e4e742eca27839bca0d440c564e506dd"
server_key_iv: "e7d3d31b8b80f1548b0e259012587037"
server_key_version: "encryptionkey@0"
I20260808 08:29:18.811903 25933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:29:18.839195 25933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:29:18.841995 25933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:29:18.842175 25933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:29:18.848840 25933 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43243
I20260808 08:29:18.848913 26038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43243 every 8 connection(s)
I20260808 08:29:18.849802 25933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:29:18.852607 26040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:29:18.855973 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 25933
I20260808 08:29:18.856127 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:29:18.856439 23471 external_mini_cluster.cc:1428] Setting key cecd68c6885213968afe6aef4ecf2cf7
I20260808 08:29:18.858628 26040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f631027ed4f848a58b97cf02ccac9616: Bootstrap starting.
I20260808 08:29:18.861110 26040 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f631027ed4f848a58b97cf02ccac9616: Neither blocks nor log segments found. Creating new log.
I20260808 08:29:18.861851 26040 log.cc:824] T 00000000000000000000000000000000 P f631027ed4f848a58b97cf02ccac9616: Log is configured to *not* fsync() on all Append() calls
I20260808 08:29:18.864377 26040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f631027ed4f848a58b97cf02ccac9616: No bootstrap required, opened a new log
Aug 08 08:29:18 dist-test-slave-f7th krb5kdc[25912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177758, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 08:29:18.867937 26040 raft_consensus.cc:348] T 00000000000000000000000000000000 P f631027ed4f848a58b97cf02ccac9616 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f631027ed4f848a58b97cf02ccac9616" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43243 } }
I20260808 08:29:18.868155 26040 raft_consensus.cc:374] T 00000000000000000000000000000000 P f631027ed4f848a58b97cf02ccac9616 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:29:18.868216 26040 raft_consensus.cc:729] T 00000000000000000000000000000000 P f631027ed4f848a58b97cf02ccac9616 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f631027ed4f848a58b97cf02ccac9616, State: Initialized, Role: FOLLOWER
I20260808 08:29:18.868774 26040 consensus_queue.cc:260] T 00000000000000000000000000000000 P f631027ed4f848a58b97cf02ccac9616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f631027ed4f848a58b97cf02ccac9616" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43243 } }
I20260808 08:29:18.868997 26040 raft_consensus.cc:388] T 00000000000000000000000000000000 P f631027ed4f848a58b97cf02ccac9616 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:29:18.869072 26040 raft_consensus.cc:482] T 00000000000000000000000000000000 P f631027ed4f848a58b97cf02ccac9616 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:29:18.869169 26040 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f631027ed4f848a58b97cf02ccac9616 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:29:18.870187 26040 raft_consensus.cc:504] T 00000000000000000000000000000000 P f631027ed4f848a58b97cf02ccac9616 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f631027ed4f848a58b97cf02ccac9616" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43243 } }
I20260808 08:29:18.870543 26040 leader_election.cc:302] T 00000000000000000000000000000000 P f631027ed4f848a58b97cf02ccac9616 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f631027ed4f848a58b97cf02ccac9616; no voters:
I20260808 08:29:18.870834 26040 leader_election.cc:288] T 00000000000000000000000000000000 P f631027ed4f848a58b97cf02ccac9616 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:29:18.870944 26050 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f631027ed4f848a58b97cf02ccac9616 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:29:18.871116 26050 raft_consensus.cc:686] T 00000000000000000000000000000000 P f631027ed4f848a58b97cf02ccac9616 [term 1 LEADER]: Becoming Leader. State: Replica: f631027ed4f848a58b97cf02ccac9616, State: Running, Role: LEADER
I20260808 08:29:18.871477 26050 consensus_queue.cc:237] T 00000000000000000000000000000000 P f631027ed4f848a58b97cf02ccac9616 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f631027ed4f848a58b97cf02ccac9616" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43243 } }
I20260808 08:29:18.872146 26040 sys_catalog.cc:564] T 00000000000000000000000000000000 P f631027ed4f848a58b97cf02ccac9616 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:29:18.872692 26050 sys_catalog.cc:455] T 00000000000000000000000000000000 P f631027ed4f848a58b97cf02ccac9616 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f631027ed4f848a58b97cf02ccac9616. Latest consensus state: current_term: 1 leader_uuid: "f631027ed4f848a58b97cf02ccac9616" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f631027ed4f848a58b97cf02ccac9616" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43243 } } }
I20260808 08:29:18.872807 26050 sys_catalog.cc:458] T 00000000000000000000000000000000 P f631027ed4f848a58b97cf02ccac9616 [sys.catalog]: This master's current role is: LEADER
I20260808 08:29:18.872910 26047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:18.857983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.209:49475 (local address 127.0.0.1:43243)
0808 08:29:18.858399 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:18.858416 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:18.858451 (+ 35us) server_negotiation.cc:407] Connection header received
0808 08:29:18.859110 (+ 659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:18.859130 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:18.859485 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:18.859760 (+ 275us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:18.860593 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:18.861642 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:18.862802 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:18.863496 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:18.863653 (+ 157us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:18.866662 (+ 3009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:18.866667 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:18.866681 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:18.866709 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:18.869410 (+ 2701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:18.869923 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:18.869929 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:18.869935 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:18.870082 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:18.870405 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:18.870409 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:18.870410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:18.870843 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:18.871148 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:18.871682 (+ 534us) server_negotiation.cc:299] Negotiation successful
0808 08:29:18.871885 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":116,"threads_started":1}
I20260808 08:29:18.873754 26052 sys_catalog.cc:455] T 00000000000000000000000000000000 P f631027ed4f848a58b97cf02ccac9616 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f631027ed4f848a58b97cf02ccac9616" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f631027ed4f848a58b97cf02ccac9616" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43243 } } }
I20260808 08:29:18.873888 26052 sys_catalog.cc:458] T 00000000000000000000000000000000 P f631027ed4f848a58b97cf02ccac9616 [sys.catalog]: This master's current role is: LEADER
I20260808 08:29:18.875975 23471 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260808 08:29:18.877221 26075 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f631027ed4f848a58b97cf02ccac9616: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 08:29:18.877331 26075 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 08:29:18.877442 26057 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:29:18.878245 26057 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:29:18.884229 26057 catalog_manager.cc:1347] Generated new cluster ID: caa33fb6dff7405dba45a06bc761b18c
I20260808 08:29:18.884294 26057 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:29:18.887703 26047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:18.878197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.209:46071 (local address 127.0.0.1:43243)
0808 08:29:18.878294 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:18.878298 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:18.878533 (+ 235us) server_negotiation.cc:407] Connection header received
0808 08:29:18.878638 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:18.878641 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:18.878688 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:18.878771 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:18.879512 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:18.880416 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:18.881579 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:18.882330 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:18.882441 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:18.883814 (+ 1373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:18.883818 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:18.883821 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:18.883849 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:18.885853 (+ 2004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:18.886379 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:18.886383 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:18.886384 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:18.886460 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:18.886797 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:18.886801 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:18.886803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:18.886987 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:18.887090 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:18.887467 (+ 377us) server_negotiation.cc:299] Negotiation successful
0808 08:29:18.887561 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 08:29:18.896353 26057 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:29:18.897249 26057 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:29:18.908874 26057 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f631027ed4f848a58b97cf02ccac9616: Generated new TSK 0
I20260808 08:29:18.909539 26057 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 08:29:18.915388 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 25933
W20260808 08:29:18.934227 25909 connection.cc:535] client connection to 127.0.0.1:43243 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43243): Connection reset by peer (error 104)
2026-08-08T08:29:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1540 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786177493204855-23471-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 08:29:18 dist-test-slave-f7th krb5kdc[26091](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 08:29:18 dist-test-slave-f7th krb5kdc[26091](info): set up 2 sockets
Aug 08 08:29:18 dist-test-slave-f7th krb5kdc[26091](info): commencing operation
krb5kdc: starting...
W20260808 08:29:20.097661 23471 mini_kdc.cc:121] Time spent starting KDC: real 1.149s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Aug 08 08:29:20 dist-test-slave-f7th krb5kdc[26091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177760, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T08:29:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:29:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786177493204855-23471-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786177493204855-23471-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 08:29:20.212328 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45025
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786177493204855-23471-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45025
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:29:20.323748 26111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:29:20.323995 26111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:29:20.324051 26111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:29:20.327986 26111 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:29:20.328042 26111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:29:20.328068 26111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:29:20.328087 26111 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:29:20.328114 26111 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:29:20.333225 26111 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45025
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786177493204855-23471-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:29:20.334192 26111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:29:20.335148 26111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:29:20.343436 26116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:20.343469 26111 server_base.cc:1034] running on GCE node
W20260808 08:29:20.343446 26117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:20.343497 26120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:20.344322 26111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:29:20.345428 26111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:29:20.346573 26111 hybrid_clock.cc:648] HybridClock initialized: now 1786177760346552 us; error 36 us; skew 500 ppm
Aug 08 08:29:20 dist-test-slave-f7th krb5kdc[26091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177760, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:29:20.349205 26111 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 08:29:20.350035 26111 webserver.cc:460] Webserver started at http://127.0.0.1:39099/ using document root <none> and password file <none>
I20260808 08:29:20.350560 26111 fs_manager.cc:359] Metadata directory not provided
I20260808 08:29:20.350625 26111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:29:20.350826 26111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:29:20.352799 26111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "81e8272b8b6447168f2fb7d041f3028b"
format_stamp: "Formatted at 2026-08-08 08:29:20 on dist-test-slave-f7th"
server_key: "8fc87f0ededda289e87282a4c5f411fb"
server_key_iv: "8e3d023d0f2a5286399665819b8f3cb4"
server_key_version: "encryptionkey@0"
I20260808 08:29:20.353312 26111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "81e8272b8b6447168f2fb7d041f3028b"
format_stamp: "Formatted at 2026-08-08 08:29:20 on dist-test-slave-f7th"
server_key: "8fc87f0ededda289e87282a4c5f411fb"
server_key_iv: "8e3d023d0f2a5286399665819b8f3cb4"
server_key_version: "encryptionkey@0"
I20260808 08:29:20.356858 26111 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 08:29:20.359077 26127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:20.360208 26111 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:29:20.360320 26111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "81e8272b8b6447168f2fb7d041f3028b"
format_stamp: "Formatted at 2026-08-08 08:29:20 on dist-test-slave-f7th"
server_key: "8fc87f0ededda289e87282a4c5f411fb"
server_key_iv: "8e3d023d0f2a5286399665819b8f3cb4"
server_key_version: "encryptionkey@0"
I20260808 08:29:20.360406 26111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:29:20.373322 26111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:29:20.375813 26111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:29:20.375973 26111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:29:20.382529 26111 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45025
I20260808 08:29:20.382570 26216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45025 every 8 connection(s)
I20260808 08:29:20.383409 26111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:29:20.386224 26217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:29:20.387974 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 26111
I20260808 08:29:20.388104 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:29:20.388396 23471 external_mini_cluster.cc:1428] Setting key a5e25524f4f788a3c258a88eefde3bd1
I20260808 08:29:20.392412 26217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81e8272b8b6447168f2fb7d041f3028b: Bootstrap starting.
I20260808 08:29:20.394584 26217 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81e8272b8b6447168f2fb7d041f3028b: Neither blocks nor log segments found. Creating new log.
I20260808 08:29:20.395277 26217 log.cc:824] T 00000000000000000000000000000000 P 81e8272b8b6447168f2fb7d041f3028b: Log is configured to *not* fsync() on all Append() calls
I20260808 08:29:20.396788 26221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:20.389793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.209:50575 (local address 127.0.0.1:45025)
0808 08:29:20.390216 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:20.390231 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:20.390265 (+ 34us) server_negotiation.cc:407] Connection header received
0808 08:29:20.390885 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:20.390901 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:20.391226 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:20.391517 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:20.391900 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:20.392899 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:20.393767 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:20.394215 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:20.394358 (+ 143us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:20.395181 (+ 823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:20.395186 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:20.395199 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:29:20.395576 (+ 377us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:20.395631 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:29:20.395653 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:20.395760 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:20.395962 (+ 202us) server_negotiation.cc:299] Negotiation successful
0808 08:29:20.396091 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":134,"threads_started":1}
I20260808 08:29:20.397169 26217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81e8272b8b6447168f2fb7d041f3028b: No bootstrap required, opened a new log
I20260808 08:29:20.399562 23471 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 08:29:20.400996 26217 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81e8272b8b6447168f2fb7d041f3028b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81e8272b8b6447168f2fb7d041f3028b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45025 } }
I20260808 08:29:20.401216 26217 raft_consensus.cc:374] T 00000000000000000000000000000000 P 81e8272b8b6447168f2fb7d041f3028b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:29:20.401274 26217 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81e8272b8b6447168f2fb7d041f3028b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81e8272b8b6447168f2fb7d041f3028b, State: Initialized, Role: FOLLOWER
I20260808 08:29:20.401701 26217 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81e8272b8b6447168f2fb7d041f3028b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81e8272b8b6447168f2fb7d041f3028b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45025 } }
I20260808 08:29:20.401835 26217 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81e8272b8b6447168f2fb7d041f3028b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:29:20.401903 26217 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81e8272b8b6447168f2fb7d041f3028b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:29:20.402014 26217 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81e8272b8b6447168f2fb7d041f3028b [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:29:20.402925 26217 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81e8272b8b6447168f2fb7d041f3028b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81e8272b8b6447168f2fb7d041f3028b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45025 } }
I20260808 08:29:20.403294 26217 leader_election.cc:302] T 00000000000000000000000000000000 P 81e8272b8b6447168f2fb7d041f3028b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81e8272b8b6447168f2fb7d041f3028b; no voters:
I20260808 08:29:20.403607 26217 leader_election.cc:288] T 00000000000000000000000000000000 P 81e8272b8b6447168f2fb7d041f3028b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:29:20.403687 26237 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81e8272b8b6447168f2fb7d041f3028b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:29:20.403918 26237 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81e8272b8b6447168f2fb7d041f3028b [term 1 LEADER]: Becoming Leader. State: Replica: 81e8272b8b6447168f2fb7d041f3028b, State: Running, Role: LEADER
I20260808 08:29:20.404241 26237 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81e8272b8b6447168f2fb7d041f3028b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81e8272b8b6447168f2fb7d041f3028b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45025 } }
I20260808 08:29:20.404934 26217 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81e8272b8b6447168f2fb7d041f3028b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:29:20.405428 26239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81e8272b8b6447168f2fb7d041f3028b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81e8272b8b6447168f2fb7d041f3028b. Latest consensus state: current_term: 1 leader_uuid: "81e8272b8b6447168f2fb7d041f3028b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81e8272b8b6447168f2fb7d041f3028b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45025 } } }
I20260808 08:29:20.405553 26239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81e8272b8b6447168f2fb7d041f3028b [sys.catalog]: This master's current role is: LEADER
I20260808 08:29:20.405835 26238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81e8272b8b6447168f2fb7d041f3028b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "81e8272b8b6447168f2fb7d041f3028b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81e8272b8b6447168f2fb7d041f3028b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45025 } } }
I20260808 08:29:20.405934 26238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81e8272b8b6447168f2fb7d041f3028b [sys.catalog]: This master's current role is: LEADER
I20260808 08:29:20.406118 26221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:20.401446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.209:58463 (local address 127.0.0.1:45025)
0808 08:29:20.401581 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:20.401586 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:20.401819 (+ 233us) server_negotiation.cc:407] Connection header received
0808 08:29:20.401919 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:20.401922 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:20.401964 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:20.402036 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:20.402352 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:20.402830 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:20.403924 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:20.404253 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:20.404328 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:20.405361 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:20.405364 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:20.405367 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:29:20.405380 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:20.405405 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:29:20.405423 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:20.405495 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:20.405745 (+ 250us) server_negotiation.cc:299] Negotiation successful
0808 08:29:20.405784 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 08:29:20.406176 26244 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:29:20.409030 26244 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:29:20.414004 26244 catalog_manager.cc:1347] Generated new cluster ID: 0644a575671e4d15af8b711649a18a8e
I20260808 08:29:20.414067 26244 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:29:20.424531 26244 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:29:20.425820 26244 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:29:20.437114 26244 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 81e8272b8b6447168f2fb7d041f3028b: Generated new TSK 0
I20260808 08:29:20.437973 26244 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 08:29:20.443228 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 26111
W20260808 08:29:20.462648 26088 connection.cc:535] client connection to 127.0.0.1:45025 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45025): Connection reset by peer (error 104)
2026-08-08T08:29:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1528 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786177493204855-23471-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 08:29:20 dist-test-slave-f7th krb5kdc[26269](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 08:29:20 dist-test-slave-f7th krb5kdc[26269](info): set up 2 sockets
Aug 08 08:29:20 dist-test-slave-f7th krb5kdc[26269](info): commencing operation
krb5kdc: starting...
W20260808 08:29:21.619844 23471 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Aug 08 08:29:21 dist-test-slave-f7th krb5kdc[26269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T08:29:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:29:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786177493204855-23471-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786177493204855-23471-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 08:29:21.737429 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44179
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786177493204855-23471-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44179
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:29:21.847668 26294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:29:21.847990 26294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:29:21.848052 26294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:29:21.852139 26294 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:29:21.852211 26294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:29:21.852237 26294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:29:21.852259 26294 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:29:21.852284 26294 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:29:21.857502 26294 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44179
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786177493204855-23471-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26294
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:29:21.858511 26294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:29:21.859499 26294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:29:21.867822 26300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:21.867851 26305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:21.867822 26301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:21.868151 26294 server_base.cc:1034] running on GCE node
I20260808 08:29:21.868732 26294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:29:21.869779 26294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:29:21.870940 26294 hybrid_clock.cc:648] HybridClock initialized: now 1786177761870924 us; error 31 us; skew 500 ppm
Aug 08 08:29:21 dist-test-slave-f7th krb5kdc[26269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177761, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:29:21.873741 26294 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 08:29:21.874790 26294 webserver.cc:460] Webserver started at http://127.0.0.1:43623/ using document root <none> and password file <none>
I20260808 08:29:21.875456 26294 fs_manager.cc:359] Metadata directory not provided
I20260808 08:29:21.875530 26294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:29:21.875782 26294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:29:21.877820 26294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "d9f4386566a34dc2adda42ca1676cd07"
format_stamp: "Formatted at 2026-08-08 08:29:21 on dist-test-slave-f7th"
server_key: "a7f602775693d1dac6901ac31caa5842"
server_key_iv: "9f350d6c7fa2c230aec5405eb1e67d76"
server_key_version: "encryptionkey@0"
I20260808 08:29:21.878351 26294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "d9f4386566a34dc2adda42ca1676cd07"
format_stamp: "Formatted at 2026-08-08 08:29:21 on dist-test-slave-f7th"
server_key: "a7f602775693d1dac6901ac31caa5842"
server_key_iv: "9f350d6c7fa2c230aec5405eb1e67d76"
server_key_version: "encryptionkey@0"
I20260808 08:29:21.881884 26294 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 08:29:21.884140 26316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:21.885294 26294 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:29:21.885407 26294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "d9f4386566a34dc2adda42ca1676cd07"
format_stamp: "Formatted at 2026-08-08 08:29:21 on dist-test-slave-f7th"
server_key: "a7f602775693d1dac6901ac31caa5842"
server_key_iv: "9f350d6c7fa2c230aec5405eb1e67d76"
server_key_version: "encryptionkey@0"
I20260808 08:29:21.885494 26294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:29:21.889827 26294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:29:21.892524 26294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:29:21.892750 26294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:29:21.899629 26294 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44179
I20260808 08:29:21.899643 26391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44179 every 8 connection(s)
I20260808 08:29:21.900548 26294 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:29:21.903144 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 26294
I20260808 08:29:21.903270 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:29:21.903331 26394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:29:21.903560 23471 external_mini_cluster.cc:1428] Setting key 8ddc285d7cb9fbf0ecba30e936807268
I20260808 08:29:21.909230 26397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:21.904943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.209:40645 (local address 127.0.0.1:44179)
0808 08:29:21.905363 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:21.905382 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:21.905424 (+ 42us) server_negotiation.cc:407] Connection header received
0808 08:29:21.906428 (+ 1004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:21.906453 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:21.906832 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:21.907202 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:21.907830 (+ 628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:21.907836 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:21.907850 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:29:21.907860 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:21.907914 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:29:21.907939 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:21.908074 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:21.908185 (+ 111us) server_negotiation.cc:299] Negotiation successful
0808 08:29:21.908347 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":98,"threads_started":1}
I20260808 08:29:21.910404 26394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9f4386566a34dc2adda42ca1676cd07: Bootstrap starting.
I20260808 08:29:21.911959 23471 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 08:29:21.913081 26394 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d9f4386566a34dc2adda42ca1676cd07: Neither blocks nor log segments found. Creating new log.
I20260808 08:29:21.913980 26394 log.cc:824] T 00000000000000000000000000000000 P d9f4386566a34dc2adda42ca1676cd07: Log is configured to *not* fsync() on all Append() calls
I20260808 08:29:21.915746 26397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:21.914229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.209:59119 (local address 127.0.0.1:44179)
0808 08:29:21.914348 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:21.914352 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:21.914582 (+ 230us) server_negotiation.cc:407] Connection header received
0808 08:29:21.914707 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:21.914710 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:21.914741 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:21.914824 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:21.915424 (+ 600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:21.915427 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:21.915429 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:29:21.915441 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:21.915467 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:29:21.915480 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:21.915562 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:21.915608 (+ 46us) server_negotiation.cc:299] Negotiation successful
0808 08:29:21.915642 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 08:29:21.916052 26394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9f4386566a34dc2adda42ca1676cd07: No bootstrap required, opened a new log
I20260808 08:29:21.919458 26394 raft_consensus.cc:348] T 00000000000000000000000000000000 P d9f4386566a34dc2adda42ca1676cd07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9f4386566a34dc2adda42ca1676cd07" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44179 } }
I20260808 08:29:21.919667 26394 raft_consensus.cc:374] T 00000000000000000000000000000000 P d9f4386566a34dc2adda42ca1676cd07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:29:21.919723 26394 raft_consensus.cc:729] T 00000000000000000000000000000000 P d9f4386566a34dc2adda42ca1676cd07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9f4386566a34dc2adda42ca1676cd07, State: Initialized, Role: FOLLOWER
I20260808 08:29:21.920133 26394 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9f4386566a34dc2adda42ca1676cd07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9f4386566a34dc2adda42ca1676cd07" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44179 } }
I20260808 08:29:21.920261 26394 raft_consensus.cc:388] T 00000000000000000000000000000000 P d9f4386566a34dc2adda42ca1676cd07 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:29:21.920326 26394 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9f4386566a34dc2adda42ca1676cd07 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:29:21.920431 26394 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9f4386566a34dc2adda42ca1676cd07 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:29:21.921273 26394 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9f4386566a34dc2adda42ca1676cd07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9f4386566a34dc2adda42ca1676cd07" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44179 } }
I20260808 08:29:21.921592 26394 leader_election.cc:302] T 00000000000000000000000000000000 P d9f4386566a34dc2adda42ca1676cd07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d9f4386566a34dc2adda42ca1676cd07; no voters:
I20260808 08:29:21.921877 26394 leader_election.cc:288] T 00000000000000000000000000000000 P d9f4386566a34dc2adda42ca1676cd07 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:29:21.921985 26412 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d9f4386566a34dc2adda42ca1676cd07 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:29:21.922248 26412 raft_consensus.cc:686] T 00000000000000000000000000000000 P d9f4386566a34dc2adda42ca1676cd07 [term 1 LEADER]: Becoming Leader. State: Replica: d9f4386566a34dc2adda42ca1676cd07, State: Running, Role: LEADER
I20260808 08:29:21.922601 26412 consensus_queue.cc:237] T 00000000000000000000000000000000 P d9f4386566a34dc2adda42ca1676cd07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9f4386566a34dc2adda42ca1676cd07" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44179 } }
I20260808 08:29:21.923081 26394 sys_catalog.cc:564] T 00000000000000000000000000000000 P d9f4386566a34dc2adda42ca1676cd07 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:29:21.924106 26413 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9f4386566a34dc2adda42ca1676cd07 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9f4386566a34dc2adda42ca1676cd07. Latest consensus state: current_term: 1 leader_uuid: "d9f4386566a34dc2adda42ca1676cd07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9f4386566a34dc2adda42ca1676cd07" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44179 } } }
I20260808 08:29:21.924233 26413 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9f4386566a34dc2adda42ca1676cd07 [sys.catalog]: This master's current role is: LEADER
I20260808 08:29:21.924657 26423 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:29:21.924595 26414 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9f4386566a34dc2adda42ca1676cd07 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d9f4386566a34dc2adda42ca1676cd07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9f4386566a34dc2adda42ca1676cd07" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44179 } } }
I20260808 08:29:21.924779 26414 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9f4386566a34dc2adda42ca1676cd07 [sys.catalog]: This master's current role is: LEADER
I20260808 08:29:21.927438 26423 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:29:21.932257 26423 catalog_manager.cc:1347] Generated new cluster ID: 4044f8d2e3b0444194349af950961af8
I20260808 08:29:21.932333 26423 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:29:21.943840 26423 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:29:21.944178 26423 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:29:21.950727 26423 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d9f4386566a34dc2adda42ca1676cd07: Generated new TSK 0
I20260808 08:29:21.951474 26423 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 08:29:21.955371 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 26294
2026-08-08T08:29:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1520 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786177493204855-23471-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 08:29:22 dist-test-slave-f7th krb5kdc[26443](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 08:29:22 dist-test-slave-f7th krb5kdc[26443](info): set up 2 sockets
Aug 08 08:29:22 dist-test-slave-f7th krb5kdc[26443](info): commencing operation
krb5kdc: starting...
W20260808 08:29:23.144730 23471 mini_kdc.cc:121] Time spent starting KDC: real 1.147s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Aug 08 08:29:23 dist-test-slave-f7th krb5kdc[26443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T08:29:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:29:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786177493204855-23471-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786177493204855-23471-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 08:29:23.267832 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41205
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786177493204855-23471-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41205
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:29:23.383881 26468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:29:23.384159 26468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:29:23.384210 26468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:29:23.388345 26468 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:29:23.388408 26468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:29:23.388427 26468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:29:23.388442 26468 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:29:23.388456 26468 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:29:23.394165 26468 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41205
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786177493204855-23471-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41205
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:29:23.395195 26468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:29:23.396178 26468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:29:23.404394 26488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:23.404482 26468 server_base.cc:1034] running on GCE node
W20260808 08:29:23.404469 26483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:23.404394 26481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:23.405210 26468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:29:23.406272 26468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:29:23.407426 26468 hybrid_clock.cc:648] HybridClock initialized: now 1786177763407374 us; error 65 us; skew 500 ppm
Aug 08 08:29:23 dist-test-slave-f7th krb5kdc[26443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177763, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:29:23.410115 26468 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 08:29:23.410970 26468 webserver.cc:460] Webserver started at http://127.0.0.1:38197/ using document root <none> and password file <none>
I20260808 08:29:23.411482 26468 fs_manager.cc:359] Metadata directory not provided
I20260808 08:29:23.411551 26468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:29:23.411751 26468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:29:23.413761 26468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "ed30d77b508e4a43afcb579aa4ff9e63"
format_stamp: "Formatted at 2026-08-08 08:29:23 on dist-test-slave-f7th"
server_key: "8d16eb3e420462f88b81f731fbb192e5"
server_key_iv: "62882d05e550a1f77684988a94f35330"
server_key_version: "encryptionkey@0"
I20260808 08:29:23.414270 26468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "ed30d77b508e4a43afcb579aa4ff9e63"
format_stamp: "Formatted at 2026-08-08 08:29:23 on dist-test-slave-f7th"
server_key: "8d16eb3e420462f88b81f731fbb192e5"
server_key_iv: "62882d05e550a1f77684988a94f35330"
server_key_version: "encryptionkey@0"
I20260808 08:29:23.417749 26468 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 08:29:23.419953 26495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:23.421128 26468 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:29:23.421233 26468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "ed30d77b508e4a43afcb579aa4ff9e63"
format_stamp: "Formatted at 2026-08-08 08:29:23 on dist-test-slave-f7th"
server_key: "8d16eb3e420462f88b81f731fbb192e5"
server_key_iv: "62882d05e550a1f77684988a94f35330"
server_key_version: "encryptionkey@0"
I20260808 08:29:23.421326 26468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:29:23.434250 26468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:29:23.436836 26468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:29:23.437034 26468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:29:23.443750 26468 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41205
I20260808 08:29:23.443769 26573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41205 every 8 connection(s)
I20260808 08:29:23.444944 26468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:29:23.445158 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 26468
I20260808 08:29:23.445297 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:29:23.445677 23471 external_mini_cluster.cc:1428] Setting key a73cc114682e48d2a1abdd1bd19bb8cf
I20260808 08:29:23.447978 26574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:29:23.453477 26574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed30d77b508e4a43afcb579aa4ff9e63: Bootstrap starting.
Aug 08 08:29:23 dist-test-slave-f7th krb5kdc[26443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 08:29:23.455996 26574 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed30d77b508e4a43afcb579aa4ff9e63: Neither blocks nor log segments found. Creating new log.
I20260808 08:29:23.456746 26574 log.cc:824] T 00000000000000000000000000000000 P ed30d77b508e4a43afcb579aa4ff9e63: Log is configured to *not* fsync() on all Append() calls
I20260808 08:29:23.458799 26574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed30d77b508e4a43afcb579aa4ff9e63: No bootstrap required, opened a new log
I20260808 08:29:23.459712 26580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:23.447019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.209:35615 (local address 127.0.0.1:41205)
0808 08:29:23.447424 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:23.447445 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:23.447651 (+ 206us) server_negotiation.cc:407] Connection header received
0808 08:29:23.448427 (+ 776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:23.448446 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:23.448837 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:23.449153 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:23.449951 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:23.450918 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:23.451777 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:23.452242 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:23.452386 (+ 144us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:23.455188 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:23.455193 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:23.455205 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:23.455233 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:23.456887 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:23.457439 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:23.457446 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:23.457451 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:23.457567 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:23.457883 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:23.457885 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:23.457886 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:23.458276 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:23.458426 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:23.458751 (+ 325us) server_negotiation.cc:299] Negotiation successful
0808 08:29:23.458959 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":106,"threads_started":1}
I20260808 08:29:23.462795 23471 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 08:29:23.462626 26574 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed30d77b508e4a43afcb579aa4ff9e63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed30d77b508e4a43afcb579aa4ff9e63" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41205 } }
I20260808 08:29:23.462869 26574 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed30d77b508e4a43afcb579aa4ff9e63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:29:23.462913 26574 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed30d77b508e4a43afcb579aa4ff9e63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed30d77b508e4a43afcb579aa4ff9e63, State: Initialized, Role: FOLLOWER
I20260808 08:29:23.463367 26574 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed30d77b508e4a43afcb579aa4ff9e63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed30d77b508e4a43afcb579aa4ff9e63" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41205 } }
I20260808 08:29:23.463519 26574 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed30d77b508e4a43afcb579aa4ff9e63 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:29:23.463588 26574 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed30d77b508e4a43afcb579aa4ff9e63 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:29:23.463682 26574 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed30d77b508e4a43afcb579aa4ff9e63 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:29:23.464776 26574 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed30d77b508e4a43afcb579aa4ff9e63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed30d77b508e4a43afcb579aa4ff9e63" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41205 } }
I20260808 08:29:23.465119 26574 leader_election.cc:302] T 00000000000000000000000000000000 P ed30d77b508e4a43afcb579aa4ff9e63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed30d77b508e4a43afcb579aa4ff9e63; no voters:
I20260808 08:29:23.465420 26574 leader_election.cc:288] T 00000000000000000000000000000000 P ed30d77b508e4a43afcb579aa4ff9e63 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:29:23.465576 26590 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed30d77b508e4a43afcb579aa4ff9e63 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:29:23.465854 26590 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed30d77b508e4a43afcb579aa4ff9e63 [term 1 LEADER]: Becoming Leader. State: Replica: ed30d77b508e4a43afcb579aa4ff9e63, State: Running, Role: LEADER
I20260808 08:29:23.466230 26590 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed30d77b508e4a43afcb579aa4ff9e63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed30d77b508e4a43afcb579aa4ff9e63" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41205 } }
I20260808 08:29:23.466668 26574 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed30d77b508e4a43afcb579aa4ff9e63 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:29:23.467434 26595 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed30d77b508e4a43afcb579aa4ff9e63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed30d77b508e4a43afcb579aa4ff9e63. Latest consensus state: current_term: 1 leader_uuid: "ed30d77b508e4a43afcb579aa4ff9e63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed30d77b508e4a43afcb579aa4ff9e63" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41205 } } }
I20260808 08:29:23.467550 26595 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed30d77b508e4a43afcb579aa4ff9e63 [sys.catalog]: This master's current role is: LEADER
I20260808 08:29:23.468222 26591 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed30d77b508e4a43afcb579aa4ff9e63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed30d77b508e4a43afcb579aa4ff9e63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed30d77b508e4a43afcb579aa4ff9e63" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41205 } } }
I20260808 08:29:23.468317 26591 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed30d77b508e4a43afcb579aa4ff9e63 [sys.catalog]: This master's current role is: LEADER
I20260808 08:29:23.468686 26604 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:29:23.472052 26604 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:29:23.473345 26580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:23.465118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.209:43729 (local address 127.0.0.1:41205)
0808 08:29:23.465245 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:23.465250 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:23.465489 (+ 239us) server_negotiation.cc:407] Connection header received
0808 08:29:23.465679 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:23.465683 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:23.465731 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:23.465812 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:23.466599 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:23.467376 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:23.468553 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:23.468901 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:23.468998 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:23.470413 (+ 1415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:23.470417 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:23.470419 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:23.470442 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:23.471699 (+ 1257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:23.472137 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:23.472140 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:23.472142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:23.472217 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:23.472520 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:23.472523 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:23.472525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:23.472738 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:23.472869 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:23.473123 (+ 254us) server_negotiation.cc:299] Negotiation successful
0808 08:29:23.473208 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 08:29:23.478051 26604 catalog_manager.cc:1347] Generated new cluster ID: d81aa643aba041a3bbc7c27ed57431b4
I20260808 08:29:23.478119 26604 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:29:23.497699 26604 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:29:23.498940 26604 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:29:23.505529 26604 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed30d77b508e4a43afcb579aa4ff9e63: Generated new TSK 0
I20260808 08:29:23.506325 26604 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 08:29:23.511127 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 26468
W20260808 08:29:23.528270 26439 connection.cc:535] client connection to 127.0.0.1:41205 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41205): Connection reset by peer (error 104)
2026-08-08T08:29:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1545 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786177493204855-23471-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 08:29:23 dist-test-slave-f7th krb5kdc[26627](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 08:29:23 dist-test-slave-f7th krb5kdc[26627](info): set up 2 sockets
Aug 08 08:29:23 dist-test-slave-f7th krb5kdc[26627](info): commencing operation
krb5kdc: starting...
W20260808 08:29:24.686589 23471 mini_kdc.cc:121] Time spent starting KDC: real 1.144s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Aug 08 08:29:24 dist-test-slave-f7th krb5kdc[26627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177764, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T08:29:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:29:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786177493204855-23471-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786177493204855-23471-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 08:29:24.811137 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33717
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786177493204855-23471-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33717
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:29:24.927029 26647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:29:24.927314 26647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:29:24.927376 26647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:29:24.931394 26647 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:29:24.931461 26647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:29:24.931490 26647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:29:24.931511 26647 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:29:24.931538 26647 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:29:24.937211 26647 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33717
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786177493204855-23471-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26647
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:29:24.938359 26647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:29:24.939427 26647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 08:29:24.948333 26647 server_base.cc:1034] running on GCE node
W20260808 08:29:24.948239 26655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:24.948239 26661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:24.948267 26656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:24.949170 26647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:29:24.950286 26647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:29:24.951447 26647 hybrid_clock.cc:648] HybridClock initialized: now 1786177764951424 us; error 36 us; skew 500 ppm
Aug 08 08:29:24 dist-test-slave-f7th krb5kdc[26627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177764, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:29:24.958750 26647 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 08:29:24.959767 26647 webserver.cc:460] Webserver started at http://127.0.0.1:43327/ using document root <none> and password file <none>
I20260808 08:29:24.960373 26647 fs_manager.cc:359] Metadata directory not provided
I20260808 08:29:24.960451 26647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:29:24.960696 26647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:29:24.962855 26647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "51a6c9dba1f04fbbb34147b4d10940a8"
format_stamp: "Formatted at 2026-08-08 08:29:24 on dist-test-slave-f7th"
server_key: "c4ae273ad0c60bdbbf2e40f0fe4922a6"
server_key_iv: "87eb0424cebb3ea497cefdc7a2f0361b"
server_key_version: "encryptionkey@0"
I20260808 08:29:24.963434 26647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "51a6c9dba1f04fbbb34147b4d10940a8"
format_stamp: "Formatted at 2026-08-08 08:29:24 on dist-test-slave-f7th"
server_key: "c4ae273ad0c60bdbbf2e40f0fe4922a6"
server_key_iv: "87eb0424cebb3ea497cefdc7a2f0361b"
server_key_version: "encryptionkey@0"
I20260808 08:29:24.967496 26647 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 08:29:24.970124 26675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:24.971366 26647 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 08:29:24.971493 26647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "51a6c9dba1f04fbbb34147b4d10940a8"
format_stamp: "Formatted at 2026-08-08 08:29:24 on dist-test-slave-f7th"
server_key: "c4ae273ad0c60bdbbf2e40f0fe4922a6"
server_key_iv: "87eb0424cebb3ea497cefdc7a2f0361b"
server_key_version: "encryptionkey@0"
I20260808 08:29:24.971604 26647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:29:24.996497 26647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:29:24.999620 26647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:29:24.999822 26647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:29:25.007040 26647 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33717
I20260808 08:29:25.007046 26757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33717 every 8 connection(s)
I20260808 08:29:25.008080 26647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:29:25.011054 26758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:29:25.016892 26758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51a6c9dba1f04fbbb34147b4d10940a8: Bootstrap starting.
I20260808 08:29:25.018065 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 26647
I20260808 08:29:25.018213 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:29:25.018523 23471 external_mini_cluster.cc:1428] Setting key ee840d10faec21f195046adad463088c
I20260808 08:29:25.019954 26758 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 51a6c9dba1f04fbbb34147b4d10940a8: Neither blocks nor log segments found. Creating new log.
I20260808 08:29:25.020754 26758 log.cc:824] T 00000000000000000000000000000000 P 51a6c9dba1f04fbbb34147b4d10940a8: Log is configured to *not* fsync() on all Append() calls
I20260808 08:29:25.022820 26758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51a6c9dba1f04fbbb34147b4d10940a8: No bootstrap required, opened a new log
I20260808 08:29:25.027107 26758 raft_consensus.cc:348] T 00000000000000000000000000000000 P 51a6c9dba1f04fbbb34147b4d10940a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51a6c9dba1f04fbbb34147b4d10940a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33717 } }
I20260808 08:29:25.027335 26758 raft_consensus.cc:374] T 00000000000000000000000000000000 P 51a6c9dba1f04fbbb34147b4d10940a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:29:25.027400 26758 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51a6c9dba1f04fbbb34147b4d10940a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51a6c9dba1f04fbbb34147b4d10940a8, State: Initialized, Role: FOLLOWER
I20260808 08:29:25.027407 26762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:25.019712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.209:45227 (local address 127.0.0.1:33717)
0808 08:29:25.020138 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:25.020154 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:25.020192 (+ 38us) server_negotiation.cc:407] Connection header received
0808 08:29:25.020925 (+ 733us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:25.020951 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:25.021320 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:25.021639 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:25.021969 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:25.022914 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:25.024096 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:25.024577 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:25.024756 (+ 179us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:25.025644 (+ 888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:25.025650 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:25.025663 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:29:25.026063 (+ 400us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:25.026120 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:29:25.026143 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:25.026245 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:25.026437 (+ 192us) server_negotiation.cc:299] Negotiation successful
0808 08:29:25.026593 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":117,"threads_started":1}
I20260808 08:29:25.027894 26758 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51a6c9dba1f04fbbb34147b4d10940a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51a6c9dba1f04fbbb34147b4d10940a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33717 } }
I20260808 08:29:25.028059 26758 raft_consensus.cc:388] T 00000000000000000000000000000000 P 51a6c9dba1f04fbbb34147b4d10940a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:29:25.028123 26758 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51a6c9dba1f04fbbb34147b4d10940a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:29:25.028218 26758 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51a6c9dba1f04fbbb34147b4d10940a8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:29:25.029201 26758 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51a6c9dba1f04fbbb34147b4d10940a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51a6c9dba1f04fbbb34147b4d10940a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33717 } }
I20260808 08:29:25.029595 26758 leader_election.cc:302] T 00000000000000000000000000000000 P 51a6c9dba1f04fbbb34147b4d10940a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 51a6c9dba1f04fbbb34147b4d10940a8; no voters:
I20260808 08:29:25.029932 26758 leader_election.cc:288] T 00000000000000000000000000000000 P 51a6c9dba1f04fbbb34147b4d10940a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:29:25.030027 26765 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 51a6c9dba1f04fbbb34147b4d10940a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:29:25.030262 26765 raft_consensus.cc:686] T 00000000000000000000000000000000 P 51a6c9dba1f04fbbb34147b4d10940a8 [term 1 LEADER]: Becoming Leader. State: Replica: 51a6c9dba1f04fbbb34147b4d10940a8, State: Running, Role: LEADER
I20260808 08:29:25.030563 23471 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 08:29:25.030647 26765 consensus_queue.cc:237] T 00000000000000000000000000000000 P 51a6c9dba1f04fbbb34147b4d10940a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51a6c9dba1f04fbbb34147b4d10940a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33717 } }
I20260808 08:29:25.031219 26758 sys_catalog.cc:564] T 00000000000000000000000000000000 P 51a6c9dba1f04fbbb34147b4d10940a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:29:25.032308 26768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51a6c9dba1f04fbbb34147b4d10940a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51a6c9dba1f04fbbb34147b4d10940a8. Latest consensus state: current_term: 1 leader_uuid: "51a6c9dba1f04fbbb34147b4d10940a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51a6c9dba1f04fbbb34147b4d10940a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33717 } } }
I20260808 08:29:25.032438 26768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51a6c9dba1f04fbbb34147b4d10940a8 [sys.catalog]: This master's current role is: LEADER
I20260808 08:29:25.032763 26767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51a6c9dba1f04fbbb34147b4d10940a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "51a6c9dba1f04fbbb34147b4d10940a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51a6c9dba1f04fbbb34147b4d10940a8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33717 } } }
I20260808 08:29:25.032850 26767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51a6c9dba1f04fbbb34147b4d10940a8 [sys.catalog]: This master's current role is: LEADER
W20260808 08:29:25.036648 26805 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 51a6c9dba1f04fbbb34147b4d10940a8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 08:29:25.036764 26805 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 08:29:25.036866 26790 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:29:25.037352 26762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:25.032975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.209:44789 (local address 127.0.0.1:33717)
0808 08:29:25.033087 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:25.033091 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:25.033293 (+ 202us) server_negotiation.cc:407] Connection header received
0808 08:29:25.033473 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:25.033476 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:25.033521 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:25.033573 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:25.033945 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:25.034535 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:25.035453 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:25.035770 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:25.035875 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:25.036827 (+ 952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:25.036830 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:25.036833 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:29:25.036846 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:25.036876 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:29:25.036890 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:25.036945 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:25.037193 (+ 248us) server_negotiation.cc:299] Negotiation successful
0808 08:29:25.037228 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 08:29:25.037752 26790 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:29:25.042778 26790 catalog_manager.cc:1347] Generated new cluster ID: 93e908fbdef94a2b8b80214b0a9e30c3
I20260808 08:29:25.042841 26790 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:29:25.068986 26790 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:29:25.070009 26790 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:29:25.074724 26790 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 51a6c9dba1f04fbbb34147b4d10940a8: Generated new TSK 0
I20260808 08:29:25.075459 26790 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 08:29:25.084985 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 26647
W20260808 08:29:25.103022 26622 connection.cc:535] client connection to 127.0.0.1:33717 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33717): Connection reset by peer (error 104)
2026-08-08T08:29:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1574 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786177493204855-23471-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 08:29:25 dist-test-slave-f7th krb5kdc[26815](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 08:29:25 dist-test-slave-f7th krb5kdc[26815](info): set up 2 sockets
Aug 08 08:29:25 dist-test-slave-f7th krb5kdc[26815](info): commencing operation
krb5kdc: starting...
W20260808 08:29:26.262768 23471 mini_kdc.cc:121] Time spent starting KDC: real 1.146s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Aug 08 08:29:26 dist-test-slave-f7th krb5kdc[26815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T08:29:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:29:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786177493204855-23471-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786177493204855-23471-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 08:29:26.381791 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33313
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786177493204855-23471-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33313
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:29:26.492355 26838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:29:26.492645 26838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:29:26.492710 26838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:29:26.496632 26838 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:29:26.496690 26838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:29:26.496716 26838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:29:26.496737 26838 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:29:26.496757 26838 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:29:26.501940 26838 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33313
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786177493204855-23471-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33313
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26838
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:29:26.502951 26838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:29:26.503950 26838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:29:26.512295 26845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:26.512266 26844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:26.512276 26838 server_base.cc:1034] running on GCE node
W20260808 08:29:26.512276 26848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:26.513092 26838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:29:26.514168 26838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:29:26.515329 26838 hybrid_clock.cc:648] HybridClock initialized: now 1786177766515311 us; error 30 us; skew 500 ppm
Aug 08 08:29:26 dist-test-slave-f7th krb5kdc[26815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177766, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:29:26.518054 26838 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 08:29:26.518937 26838 webserver.cc:460] Webserver started at http://127.0.0.1:45455/ using document root <none> and password file <none>
I20260808 08:29:26.519476 26838 fs_manager.cc:359] Metadata directory not provided
I20260808 08:29:26.519538 26838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:29:26.519737 26838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:29:26.521754 26838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "40bdaf5ee9854122b32aa1854cbf2349"
format_stamp: "Formatted at 2026-08-08 08:29:26 on dist-test-slave-f7th"
server_key: "51de43a2b9667f7a52c6021e06925d25"
server_key_iv: "d75c55e7e34ae6d7074c0c6dc4bea6fe"
server_key_version: "encryptionkey@0"
I20260808 08:29:26.522305 26838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "40bdaf5ee9854122b32aa1854cbf2349"
format_stamp: "Formatted at 2026-08-08 08:29:26 on dist-test-slave-f7th"
server_key: "51de43a2b9667f7a52c6021e06925d25"
server_key_iv: "d75c55e7e34ae6d7074c0c6dc4bea6fe"
server_key_version: "encryptionkey@0"
I20260808 08:29:26.525851 26838 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 08:29:26.528185 26862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:26.529378 26838 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:29:26.529500 26838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "40bdaf5ee9854122b32aa1854cbf2349"
format_stamp: "Formatted at 2026-08-08 08:29:26 on dist-test-slave-f7th"
server_key: "51de43a2b9667f7a52c6021e06925d25"
server_key_iv: "d75c55e7e34ae6d7074c0c6dc4bea6fe"
server_key_version: "encryptionkey@0"
I20260808 08:29:26.529604 26838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:29:26.534097 26838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:29:26.536777 26838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:29:26.536940 26838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:29:26.543478 26838 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33313
I20260808 08:29:26.543486 26947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33313 every 8 connection(s)
I20260808 08:29:26.544346 26838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:29:26.547073 26948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:29:26.547384 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 26838
I20260808 08:29:26.547523 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:29:26.547777 23471 external_mini_cluster.cc:1428] Setting key 7bf46988934c555078ec28342cb8770f
I20260808 08:29:26.552891 26954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:26.549127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.209:48247 (local address 127.0.0.1:33313)
0808 08:29:26.549543 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:26.549557 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:26.549593 (+ 36us) server_negotiation.cc:407] Connection header received
0808 08:29:26.550241 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:26.550257 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:26.550570 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:26.550895 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:26.551699 (+ 804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:26.551704 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:26.551716 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:29:26.551727 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:26.551780 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:29:26.551798 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:26.551922 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:26.552021 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 08:29:26.552180 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":124,"threads_started":1}
I20260808 08:29:26.553651 26948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40bdaf5ee9854122b32aa1854cbf2349: Bootstrap starting.
I20260808 08:29:26.555527 23471 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 08:29:26.555935 26948 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 40bdaf5ee9854122b32aa1854cbf2349: Neither blocks nor log segments found. Creating new log.
I20260808 08:29:26.556726 26948 log.cc:824] T 00000000000000000000000000000000 P 40bdaf5ee9854122b32aa1854cbf2349: Log is configured to *not* fsync() on all Append() calls
I20260808 08:29:26.558880 26948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40bdaf5ee9854122b32aa1854cbf2349: No bootstrap required, opened a new log
I20260808 08:29:26.559091 26954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:26.557546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.209:41633 (local address 127.0.0.1:33313)
0808 08:29:26.557666 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:26.557670 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:26.557855 (+ 185us) server_negotiation.cc:407] Connection header received
0808 08:29:26.557971 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:26.557973 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:26.558002 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:26.558063 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:26.558710 (+ 647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:26.558713 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:26.558715 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:29:26.558726 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:26.558751 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:29:26.558761 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:26.558822 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:26.558919 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 08:29:26.558955 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 08:29:26.562582 26948 raft_consensus.cc:348] T 00000000000000000000000000000000 P 40bdaf5ee9854122b32aa1854cbf2349 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40bdaf5ee9854122b32aa1854cbf2349" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33313 } }
I20260808 08:29:26.562798 26948 raft_consensus.cc:374] T 00000000000000000000000000000000 P 40bdaf5ee9854122b32aa1854cbf2349 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:29:26.562848 26948 raft_consensus.cc:729] T 00000000000000000000000000000000 P 40bdaf5ee9854122b32aa1854cbf2349 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40bdaf5ee9854122b32aa1854cbf2349, State: Initialized, Role: FOLLOWER
I20260808 08:29:26.563251 26948 consensus_queue.cc:260] T 00000000000000000000000000000000 P 40bdaf5ee9854122b32aa1854cbf2349 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40bdaf5ee9854122b32aa1854cbf2349" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33313 } }
I20260808 08:29:26.563369 26948 raft_consensus.cc:388] T 00000000000000000000000000000000 P 40bdaf5ee9854122b32aa1854cbf2349 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:29:26.563431 26948 raft_consensus.cc:482] T 00000000000000000000000000000000 P 40bdaf5ee9854122b32aa1854cbf2349 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:29:26.563532 26948 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 40bdaf5ee9854122b32aa1854cbf2349 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:29:26.564400 26948 raft_consensus.cc:504] T 00000000000000000000000000000000 P 40bdaf5ee9854122b32aa1854cbf2349 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40bdaf5ee9854122b32aa1854cbf2349" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33313 } }
I20260808 08:29:26.564795 26948 leader_election.cc:302] T 00000000000000000000000000000000 P 40bdaf5ee9854122b32aa1854cbf2349 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 40bdaf5ee9854122b32aa1854cbf2349; no voters:
I20260808 08:29:26.565132 26948 leader_election.cc:288] T 00000000000000000000000000000000 P 40bdaf5ee9854122b32aa1854cbf2349 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:29:26.565236 26967 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 40bdaf5ee9854122b32aa1854cbf2349 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:29:26.565440 26967 raft_consensus.cc:686] T 00000000000000000000000000000000 P 40bdaf5ee9854122b32aa1854cbf2349 [term 1 LEADER]: Becoming Leader. State: Replica: 40bdaf5ee9854122b32aa1854cbf2349, State: Running, Role: LEADER
I20260808 08:29:26.565765 26967 consensus_queue.cc:237] T 00000000000000000000000000000000 P 40bdaf5ee9854122b32aa1854cbf2349 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40bdaf5ee9854122b32aa1854cbf2349" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33313 } }
I20260808 08:29:26.566339 26948 sys_catalog.cc:564] T 00000000000000000000000000000000 P 40bdaf5ee9854122b32aa1854cbf2349 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:29:26.567456 26968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40bdaf5ee9854122b32aa1854cbf2349 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 40bdaf5ee9854122b32aa1854cbf2349. Latest consensus state: current_term: 1 leader_uuid: "40bdaf5ee9854122b32aa1854cbf2349" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40bdaf5ee9854122b32aa1854cbf2349" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33313 } } }
I20260808 08:29:26.567576 26968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40bdaf5ee9854122b32aa1854cbf2349 [sys.catalog]: This master's current role is: LEADER
I20260808 08:29:26.567878 26969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40bdaf5ee9854122b32aa1854cbf2349 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "40bdaf5ee9854122b32aa1854cbf2349" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40bdaf5ee9854122b32aa1854cbf2349" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33313 } } }
I20260808 08:29:26.567971 26981 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:29:26.568037 26969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40bdaf5ee9854122b32aa1854cbf2349 [sys.catalog]: This master's current role is: LEADER
I20260808 08:29:26.571277 26981 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:29:26.576747 26981 catalog_manager.cc:1347] Generated new cluster ID: ff41952128784216916138ddfebbc691
I20260808 08:29:26.576802 26981 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:29:26.592698 26981 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:29:26.592996 26981 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:29:26.598455 26981 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 40bdaf5ee9854122b32aa1854cbf2349: Generated new TSK 0
I20260808 08:29:26.599112 26981 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 08:29:26.601915 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 26838
2026-08-08T08:29:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1509 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786177493204855-23471-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 08:29:26 dist-test-slave-f7th krb5kdc[26999](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 08:29:26 dist-test-slave-f7th krb5kdc[26999](info): set up 2 sockets
Aug 08 08:29:26 dist-test-slave-f7th krb5kdc[26999](info): commencing operation
krb5kdc: starting...
W20260808 08:29:27.769496 23471 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Aug 08 08:29:27 dist-test-slave-f7th krb5kdc[26999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T08:29:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:29:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.235.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
I20260808 08:29:27.891234 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:39699
--webserver_interface=127.22.235.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:35753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:39699
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:29:28.003932 27025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:29:28.004241 27025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:29:28.004304 27025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:29:28.008340 27025 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:29:28.008411 27025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:29:28.008438 27025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:29:28.008459 27025 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:29:28.008479 27025 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:29:28.013813 27025 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:35753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:39699
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:39699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.235.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27025
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:29:28.014905 27025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:29:28.015949 27025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:29:28.024427 27036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:28.024451 27035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:28.024534 27025 server_base.cc:1034] running on GCE node
W20260808 08:29:28.024446 27039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:28.025233 27025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:29:28.026321 27025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:29:28.027499 27025 hybrid_clock.cc:648] HybridClock initialized: now 1786177768027466 us; error 40 us; skew 500 ppm
Aug 08 08:29:28 dist-test-slave-f7th krb5kdc[26999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177768, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:29:28.030236 27025 init.cc:377] Logged in from keytab as kudu/127.22.235.254@KRBTEST.COM (short username kudu)
I20260808 08:29:28.031128 27025 webserver.cc:460] Webserver started at http://127.22.235.254:40313/ using document root <none> and password file <none>
I20260808 08:29:28.031675 27025 fs_manager.cc:359] Metadata directory not provided
I20260808 08:29:28.031744 27025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:29:28.031935 27025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:29:28.033954 27025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "bd83743d77854558a1f9e606dde07bda"
format_stamp: "Formatted at 2026-08-08 08:29:28 on dist-test-slave-f7th"
server_key: "1ead952fb2cf2fa8476d1c9b30cfe7c2"
server_key_iv: "3cae910a1ad7a0961302b4e7aa8805bd"
server_key_version: "encryptionkey@0"
I20260808 08:29:28.034469 27025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "bd83743d77854558a1f9e606dde07bda"
format_stamp: "Formatted at 2026-08-08 08:29:28 on dist-test-slave-f7th"
server_key: "1ead952fb2cf2fa8476d1c9b30cfe7c2"
server_key_iv: "3cae910a1ad7a0961302b4e7aa8805bd"
server_key_version: "encryptionkey@0"
I20260808 08:29:28.038236 27025 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260808 08:29:28.040527 27050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:28.041708 27025 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 08:29:28.041818 27025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "bd83743d77854558a1f9e606dde07bda"
format_stamp: "Formatted at 2026-08-08 08:29:28 on dist-test-slave-f7th"
server_key: "1ead952fb2cf2fa8476d1c9b30cfe7c2"
server_key_iv: "3cae910a1ad7a0961302b4e7aa8805bd"
server_key_version: "encryptionkey@0"
I20260808 08:29:28.041908 27025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:29:28.059976 27025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:29:28.063046 27025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:29:28.063222 27025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:29:28.070076 27025 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.254:39699
I20260808 08:29:28.070099 27138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.254:39699 every 8 connection(s)
I20260808 08:29:28.071002 27025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:29:28.073822 27141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:29:28.077404 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 27025
I20260808 08:29:28.077554 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:29:28.077854 23471 external_mini_cluster.cc:1428] Setting key 3487bf0598e505826d4736b11ae5cde8
I20260808 08:29:28.079814 27141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd83743d77854558a1f9e606dde07bda: Bootstrap starting.
I20260808 08:29:28.082010 27141 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd83743d77854558a1f9e606dde07bda: Neither blocks nor log segments found. Creating new log.
I20260808 08:29:28.082762 27141 log.cc:824] T 00000000000000000000000000000000 P bd83743d77854558a1f9e606dde07bda: Log is configured to *not* fsync() on all Append() calls
I20260808 08:29:28.084537 27141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd83743d77854558a1f9e606dde07bda: No bootstrap required, opened a new log
Aug 08 08:29:28 dist-test-slave-f7th krb5kdc[26999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:29:28.088167 27141 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd83743d77854558a1f9e606dde07bda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd83743d77854558a1f9e606dde07bda" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 39699 } }
I20260808 08:29:28.088383 27141 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd83743d77854558a1f9e606dde07bda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:29:28.088429 27141 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd83743d77854558a1f9e606dde07bda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd83743d77854558a1f9e606dde07bda, State: Initialized, Role: FOLLOWER
I20260808 08:29:28.088882 27141 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd83743d77854558a1f9e606dde07bda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd83743d77854558a1f9e606dde07bda" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 39699 } }
I20260808 08:29:28.089015 27141 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd83743d77854558a1f9e606dde07bda [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:29:28.089075 27141 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd83743d77854558a1f9e606dde07bda [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:29:28.089180 27141 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd83743d77854558a1f9e606dde07bda [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:29:28.090027 27141 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd83743d77854558a1f9e606dde07bda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd83743d77854558a1f9e606dde07bda" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 39699 } }
I20260808 08:29:28.090353 27141 leader_election.cc:302] T 00000000000000000000000000000000 P bd83743d77854558a1f9e606dde07bda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd83743d77854558a1f9e606dde07bda; no voters:
I20260808 08:29:28.090679 27141 leader_election.cc:288] T 00000000000000000000000000000000 P bd83743d77854558a1f9e606dde07bda [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:29:28.090781 27146 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd83743d77854558a1f9e606dde07bda [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:29:28.090994 27146 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd83743d77854558a1f9e606dde07bda [term 1 LEADER]: Becoming Leader. State: Replica: bd83743d77854558a1f9e606dde07bda, State: Running, Role: LEADER
I20260808 08:29:28.091352 27146 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd83743d77854558a1f9e606dde07bda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd83743d77854558a1f9e606dde07bda" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 39699 } }
I20260808 08:29:28.091730 27144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:28.079307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52527 (local address 127.22.235.254:39699)
0808 08:29:28.079716 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:28.079731 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:28.079765 (+ 34us) server_negotiation.cc:407] Connection header received
0808 08:29:28.080394 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:28.080411 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:28.080789 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:28.081080 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:28.081875 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:28.082962 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:28.083785 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:28.084244 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:28.084387 (+ 143us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:28.086996 (+ 2609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:28.087002 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:28.087015 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:28.087042 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:28.088664 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:28.089155 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:28.089162 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:28.089168 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:28.089316 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:28.089632 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:28.089636 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:28.089638 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:28.090017 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:28.090218 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:28.090544 (+ 326us) server_negotiation.cc:299] Negotiation successful
0808 08:29:28.090782 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":130,"threads_started":1}
I20260808 08:29:28.092054 27141 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd83743d77854558a1f9e606dde07bda [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:29:28.093784 27149 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd83743d77854558a1f9e606dde07bda [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd83743d77854558a1f9e606dde07bda. Latest consensus state: current_term: 1 leader_uuid: "bd83743d77854558a1f9e606dde07bda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd83743d77854558a1f9e606dde07bda" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 39699 } } }
I20260808 08:29:28.093922 27149 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd83743d77854558a1f9e606dde07bda [sys.catalog]: This master's current role is: LEADER
I20260808 08:29:28.094285 27174 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:29:28.094275 27152 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd83743d77854558a1f9e606dde07bda [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bd83743d77854558a1f9e606dde07bda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd83743d77854558a1f9e606dde07bda" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 39699 } } }
I20260808 08:29:28.094970 27152 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd83743d77854558a1f9e606dde07bda [sys.catalog]: This master's current role is: LEADER
I20260808 08:29:28.095425 23471 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 08:29:28.098043 27174 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:29:28.104166 27174 catalog_manager.cc:1347] Generated new cluster ID: 09e00c7b655f4f9085e3f5cf30c6cd07
I20260808 08:29:28.104233 27174 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:29:28.105816 27144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:28.097644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59059 (local address 127.22.235.254:39699)
0808 08:29:28.097742 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:28.097746 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:28.097976 (+ 230us) server_negotiation.cc:407] Connection header received
0808 08:29:28.098115 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:28.098119 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:28.098167 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:28.098249 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:28.099057 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:28.099796 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:28.100588 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:28.100870 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:28.100983 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:28.102459 (+ 1476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:28.102462 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:28.102465 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:28.102498 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:28.104061 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:28.104578 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:28.104580 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:28.104581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:28.104693 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:28.105005 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:28.105009 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:28.105010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:28.105177 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:28.105304 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:28.105584 (+ 280us) server_negotiation.cc:299] Negotiation successful
0808 08:29:28.105672 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 08:29:28.121826 27174 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:29:28.122938 27174 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:29:28.128369 27174 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bd83743d77854558a1f9e606dde07bda: Generated new TSK 0
I20260808 08:29:28.129221 27174 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 08:29:28.140341 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 27025
W20260808 08:29:28.158943 26997 connection.cc:535] client connection to 127.22.235.254:39699 recv error: Network error: failed to read from TLS socket (remote: 127.22.235.254:39699): Connection reset by peer (error 104)
2026-08-08T08:29:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1545 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786177493204855-23471-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 08:29:28 dist-test-slave-f7th krb5kdc[27188](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 08:29:28 dist-test-slave-f7th krb5kdc[27188](info): set up 2 sockets
Aug 08 08:29:28 dist-test-slave-f7th krb5kdc[27188](info): commencing operation
krb5kdc: starting...
W20260808 08:29:29.310025 23471 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Aug 08 08:29:29 dist-test-slave-f7th krb5kdc[27188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T08:29:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:29:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.235.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
I20260808 08:29:29.429280 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:32893
--webserver_interface=127.22.235.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:42237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:32893
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:29:29.554529 27213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:29:29.554781 27213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:29:29.554853 27213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:29:29.558851 27213 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:29:29.558912 27213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:29:29.558938 27213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:29:29.558959 27213 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:29:29.558980 27213 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:29:29.564105 27213 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:42237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:32893
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:32893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.235.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27213
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:29:29.565178 27213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:29:29.566212 27213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:29:29.574466 27224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:29.574478 27222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:29.574518 27213 server_base.cc:1034] running on GCE node
W20260808 08:29:29.574482 27227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:29.575246 27213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:29:29.576390 27213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:29:29.577555 27213 hybrid_clock.cc:648] HybridClock initialized: now 1786177769577533 us; error 33 us; skew 500 ppm
Aug 08 08:29:29 dist-test-slave-f7th krb5kdc[27188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177769, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:29:29.580322 27213 init.cc:377] Logged in from keytab as kudu/127.22.235.254@KRBTEST.COM (short username kudu)
I20260808 08:29:29.581306 27213 webserver.cc:460] Webserver started at http://127.22.235.254:44167/ using document root <none> and password file <none>
I20260808 08:29:29.581858 27213 fs_manager.cc:359] Metadata directory not provided
I20260808 08:29:29.581926 27213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:29:29.582132 27213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:29:29.584141 27213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "5cfddf8f1e1c4afa81f596b09d81a595"
format_stamp: "Formatted at 2026-08-08 08:29:29 on dist-test-slave-f7th"
server_key: "609f8c33700329b2ac319bd7b404c21b"
server_key_iv: "87af89e4da7002d56f01743729e97fc9"
server_key_version: "encryptionkey@0"
I20260808 08:29:29.584710 27213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "5cfddf8f1e1c4afa81f596b09d81a595"
format_stamp: "Formatted at 2026-08-08 08:29:29 on dist-test-slave-f7th"
server_key: "609f8c33700329b2ac319bd7b404c21b"
server_key_iv: "87af89e4da7002d56f01743729e97fc9"
server_key_version: "encryptionkey@0"
I20260808 08:29:29.588410 27213 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 08:29:29.590759 27238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:29.591923 27213 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 08:29:29.592039 27213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "5cfddf8f1e1c4afa81f596b09d81a595"
format_stamp: "Formatted at 2026-08-08 08:29:29 on dist-test-slave-f7th"
server_key: "609f8c33700329b2ac319bd7b404c21b"
server_key_iv: "87af89e4da7002d56f01743729e97fc9"
server_key_version: "encryptionkey@0"
I20260808 08:29:29.592125 27213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:29:29.615245 27213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:29:29.618311 27213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:29:29.618485 27213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:29:29.625456 27213 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.254:32893
I20260808 08:29:29.625468 27327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.254:32893 every 8 connection(s)
I20260808 08:29:29.626395 27213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:29:29.626765 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 27213
I20260808 08:29:29.626931 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:29:29.627220 23471 external_mini_cluster.cc:1428] Setting key 4ab5a6195a290398861bb1fd9e2ee831
I20260808 08:29:29.629689 27329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:29:29.636536 27329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cfddf8f1e1c4afa81f596b09d81a595: Bootstrap starting.
I20260808 08:29:29.637061 27336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:29.628563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41069 (local address 127.22.235.254:32893)
0808 08:29:29.629051 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:29.629070 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:29.629113 (+ 43us) server_negotiation.cc:407] Connection header received
0808 08:29:29.629932 (+ 819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:29.629955 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:29.630361 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:29.630722 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:29.631101 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:29.632656 (+ 1555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:29.633507 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:29.634181 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:29.634344 (+ 163us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:29.635126 (+ 782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:29.635133 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:29.635149 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:29:29.635629 (+ 480us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:29.635696 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:29:29.635725 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:29.635866 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:29.636014 (+ 148us) server_negotiation.cc:299] Negotiation successful
0808 08:29:29.636142 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":149,"threads_started":1}
I20260808 08:29:29.639063 27329 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5cfddf8f1e1c4afa81f596b09d81a595: Neither blocks nor log segments found. Creating new log.
I20260808 08:29:29.639940 27329 log.cc:824] T 00000000000000000000000000000000 P 5cfddf8f1e1c4afa81f596b09d81a595: Log is configured to *not* fsync() on all Append() calls
I20260808 08:29:29.640029 23471 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 08:29:29.641929 27329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cfddf8f1e1c4afa81f596b09d81a595: No bootstrap required, opened a new log
I20260808 08:29:29.645828 27336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:29.642065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42375 (local address 127.22.235.254:32893)
0808 08:29:29.642179 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:29.642184 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:29.642390 (+ 206us) server_negotiation.cc:407] Connection header received
0808 08:29:29.642469 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:29.642471 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:29.642513 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:29.642586 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:29.642800 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:29.643529 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:29.644283 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:29.644600 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:29.644732 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:29.645415 (+ 683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:29.645419 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:29.645422 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:29:29.645434 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:29.645461 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:29:29.645474 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:29.645560 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:29.645658 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 08:29:29.645693 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 08:29:29.646106 27329 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5cfddf8f1e1c4afa81f596b09d81a595 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cfddf8f1e1c4afa81f596b09d81a595" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 32893 } }
I20260808 08:29:29.646368 27329 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5cfddf8f1e1c4afa81f596b09d81a595 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:29:29.646440 27329 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5cfddf8f1e1c4afa81f596b09d81a595 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cfddf8f1e1c4afa81f596b09d81a595, State: Initialized, Role: FOLLOWER
I20260808 08:29:29.646886 27329 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5cfddf8f1e1c4afa81f596b09d81a595 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cfddf8f1e1c4afa81f596b09d81a595" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 32893 } }
I20260808 08:29:29.647015 27329 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5cfddf8f1e1c4afa81f596b09d81a595 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:29:29.647060 27329 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5cfddf8f1e1c4afa81f596b09d81a595 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:29:29.647130 27329 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5cfddf8f1e1c4afa81f596b09d81a595 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:29:29.647940 27329 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5cfddf8f1e1c4afa81f596b09d81a595 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cfddf8f1e1c4afa81f596b09d81a595" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 32893 } }
I20260808 08:29:29.648226 27329 leader_election.cc:302] T 00000000000000000000000000000000 P 5cfddf8f1e1c4afa81f596b09d81a595 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5cfddf8f1e1c4afa81f596b09d81a595; no voters:
I20260808 08:29:29.648470 27329 leader_election.cc:288] T 00000000000000000000000000000000 P 5cfddf8f1e1c4afa81f596b09d81a595 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:29:29.648581 27346 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5cfddf8f1e1c4afa81f596b09d81a595 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:29:29.648811 27346 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5cfddf8f1e1c4afa81f596b09d81a595 [term 1 LEADER]: Becoming Leader. State: Replica: 5cfddf8f1e1c4afa81f596b09d81a595, State: Running, Role: LEADER
I20260808 08:29:29.649163 27346 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5cfddf8f1e1c4afa81f596b09d81a595 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cfddf8f1e1c4afa81f596b09d81a595" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 32893 } }
I20260808 08:29:29.649619 27329 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5cfddf8f1e1c4afa81f596b09d81a595 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:29:29.650645 27350 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cfddf8f1e1c4afa81f596b09d81a595 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5cfddf8f1e1c4afa81f596b09d81a595. Latest consensus state: current_term: 1 leader_uuid: "5cfddf8f1e1c4afa81f596b09d81a595" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cfddf8f1e1c4afa81f596b09d81a595" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 32893 } } }
I20260808 08:29:29.650776 27350 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cfddf8f1e1c4afa81f596b09d81a595 [sys.catalog]: This master's current role is: LEADER
I20260808 08:29:29.651170 27364 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:29:29.651862 27347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cfddf8f1e1c4afa81f596b09d81a595 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5cfddf8f1e1c4afa81f596b09d81a595" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cfddf8f1e1c4afa81f596b09d81a595" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 32893 } } }
I20260808 08:29:29.651961 27347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cfddf8f1e1c4afa81f596b09d81a595 [sys.catalog]: This master's current role is: LEADER
I20260808 08:29:29.653867 27364 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:29:29.658854 27364 catalog_manager.cc:1347] Generated new cluster ID: c5ecafca0e4d49989a867f14dee8a094
I20260808 08:29:29.658955 27364 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:29:29.670509 27364 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:29:29.671463 27364 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:29:29.685173 27364 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5cfddf8f1e1c4afa81f596b09d81a595: Generated new TSK 0
I20260808 08:29:29.685880 27364 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 08:29:29.692493 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 27213
W20260808 08:29:29.711341 27186 connection.cc:535] client connection to 127.22.235.254:32893 recv error: Network error: failed to read from TLS socket (remote: 127.22.235.254:32893): Connection reset by peer (error 104)
2026-08-08T08:29:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1552 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786177493204855-23471-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 08:29:29 dist-test-slave-f7th krb5kdc[27390](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 08:29:29 dist-test-slave-f7th krb5kdc[27390](info): set up 2 sockets
Aug 08 08:29:29 dist-test-slave-f7th krb5kdc[27390](info): commencing operation
krb5kdc: starting...
W20260808 08:29:30.876211 23471 mini_kdc.cc:121] Time spent starting KDC: real 1.151s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Aug 08 08:29:30 dist-test-slave-f7th krb5kdc[27390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177770, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T08:29:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:29:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.235.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
I20260808 08:29:30.997625 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:35997
--webserver_interface=127.22.235.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:35871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:35997
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:29:31.108807 27409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:29:31.109071 27409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:29:31.109113 27409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:29:31.113091 27409 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:29:31.113150 27409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:29:31.113168 27409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:29:31.113184 27409 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:29:31.113202 27409 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:29:31.118373 27409 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:35871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:35997
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:35997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.235.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:29:31.119354 27409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:29:31.120311 27409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:29:31.128424 27418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:31.128415 27417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:31.128453 27420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:31.128584 27409 server_base.cc:1034] running on GCE node
I20260808 08:29:31.129261 27409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:29:31.130424 27409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:29:31.131588 27409 hybrid_clock.cc:648] HybridClock initialized: now 1786177771131564 us; error 34 us; skew 500 ppm
Aug 08 08:29:31 dist-test-slave-f7th krb5kdc[27390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177771, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:29:31.140326 27409 init.cc:377] Logged in from keytab as kudu/127.22.235.254@KRBTEST.COM (short username kudu)
I20260808 08:29:31.141264 27409 webserver.cc:460] Webserver started at http://127.22.235.254:41163/ using document root <none> and password file <none>
I20260808 08:29:31.141813 27409 fs_manager.cc:359] Metadata directory not provided
I20260808 08:29:31.141881 27409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:29:31.142083 27409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:29:31.144104 27409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "83d3897dd1874a22bb7551e08ad6b4af"
format_stamp: "Formatted at 2026-08-08 08:29:31 on dist-test-slave-f7th"
server_key: "b28c2c41169272fa5a6d10f85534b7b2"
server_key_iv: "889f01b2a6e086f7a9e1a8ceb81a874b"
server_key_version: "encryptionkey@0"
I20260808 08:29:31.144677 27409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "83d3897dd1874a22bb7551e08ad6b4af"
format_stamp: "Formatted at 2026-08-08 08:29:31 on dist-test-slave-f7th"
server_key: "b28c2c41169272fa5a6d10f85534b7b2"
server_key_iv: "889f01b2a6e086f7a9e1a8ceb81a874b"
server_key_version: "encryptionkey@0"
I20260808 08:29:31.148243 27409 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 08:29:31.150625 27431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:31.151757 27409 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 08:29:31.151875 27409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "83d3897dd1874a22bb7551e08ad6b4af"
format_stamp: "Formatted at 2026-08-08 08:29:31 on dist-test-slave-f7th"
server_key: "b28c2c41169272fa5a6d10f85534b7b2"
server_key_iv: "889f01b2a6e086f7a9e1a8ceb81a874b"
server_key_version: "encryptionkey@0"
I20260808 08:29:31.151966 27409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:29:31.156369 27409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:29:31.159088 27409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:29:31.159250 27409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:29:31.166121 27409 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.254:35997
I20260808 08:29:31.166152 27529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.254:35997 every 8 connection(s)
I20260808 08:29:31.167073 27409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:29:31.169853 27530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:29:31.173779 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 27409
I20260808 08:29:31.173938 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:29:31.174234 23471 external_mini_cluster.cc:1428] Setting key 98a6066b3cb858d070473ad27f1e9d98
I20260808 08:29:31.175733 27530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83d3897dd1874a22bb7551e08ad6b4af: Bootstrap starting.
I20260808 08:29:31.178109 27530 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83d3897dd1874a22bb7551e08ad6b4af: Neither blocks nor log segments found. Creating new log.
I20260808 08:29:31.178901 27530 log.cc:824] T 00000000000000000000000000000000 P 83d3897dd1874a22bb7551e08ad6b4af: Log is configured to *not* fsync() on all Append() calls
I20260808 08:29:31.179286 27536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:31.175562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52993 (local address 127.22.235.254:35997)
0808 08:29:31.175974 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:31.175990 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:31.176025 (+ 35us) server_negotiation.cc:407] Connection header received
0808 08:29:31.176687 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:31.176706 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:31.176994 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:31.177264 (+ 270us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:31.178072 (+ 808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:31.178077 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:31.178089 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:29:31.178099 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:31.178152 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:29:31.178171 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:31.178274 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:31.178387 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 08:29:31.178537 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":117,"threads_started":1}
I20260808 08:29:31.180716 27530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83d3897dd1874a22bb7551e08ad6b4af: No bootstrap required, opened a new log
I20260808 08:29:31.181949 23471 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 08:29:31.184958 27530 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83d3897dd1874a22bb7551e08ad6b4af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83d3897dd1874a22bb7551e08ad6b4af" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 35997 } }
I20260808 08:29:31.185232 27530 raft_consensus.cc:374] T 00000000000000000000000000000000 P 83d3897dd1874a22bb7551e08ad6b4af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:29:31.185294 27530 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83d3897dd1874a22bb7551e08ad6b4af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83d3897dd1874a22bb7551e08ad6b4af, State: Initialized, Role: FOLLOWER
I20260808 08:29:31.185740 27530 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83d3897dd1874a22bb7551e08ad6b4af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83d3897dd1874a22bb7551e08ad6b4af" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 35997 } }
I20260808 08:29:31.185835 27536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:31.184283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38395 (local address 127.22.235.254:35997)
0808 08:29:31.184447 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:31.184450 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:31.184533 (+ 83us) server_negotiation.cc:407] Connection header received
0808 08:29:31.184682 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:31.184684 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:31.184713 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:31.184768 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:31.185414 (+ 646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:31.185417 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:31.185420 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:29:31.185432 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:31.185456 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:29:31.185482 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:31.185568 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:31.185658 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 08:29:31.185689 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260808 08:29:31.185914 27530 raft_consensus.cc:388] T 00000000000000000000000000000000 P 83d3897dd1874a22bb7551e08ad6b4af [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:29:31.185977 27530 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83d3897dd1874a22bb7551e08ad6b4af [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:29:31.186079 27530 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83d3897dd1874a22bb7551e08ad6b4af [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:29:31.187073 27530 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83d3897dd1874a22bb7551e08ad6b4af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83d3897dd1874a22bb7551e08ad6b4af" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 35997 } }
I20260808 08:29:31.187384 27530 leader_election.cc:302] T 00000000000000000000000000000000 P 83d3897dd1874a22bb7551e08ad6b4af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 83d3897dd1874a22bb7551e08ad6b4af; no voters:
I20260808 08:29:31.187649 27530 leader_election.cc:288] T 00000000000000000000000000000000 P 83d3897dd1874a22bb7551e08ad6b4af [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:29:31.187723 27546 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83d3897dd1874a22bb7551e08ad6b4af [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:29:31.187923 27546 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83d3897dd1874a22bb7551e08ad6b4af [term 1 LEADER]: Becoming Leader. State: Replica: 83d3897dd1874a22bb7551e08ad6b4af, State: Running, Role: LEADER
I20260808 08:29:31.188287 27546 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83d3897dd1874a22bb7551e08ad6b4af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83d3897dd1874a22bb7551e08ad6b4af" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 35997 } }
I20260808 08:29:31.188808 27530 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83d3897dd1874a22bb7551e08ad6b4af [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:29:31.190026 27548 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83d3897dd1874a22bb7551e08ad6b4af [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83d3897dd1874a22bb7551e08ad6b4af. Latest consensus state: current_term: 1 leader_uuid: "83d3897dd1874a22bb7551e08ad6b4af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83d3897dd1874a22bb7551e08ad6b4af" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 35997 } } }
I20260808 08:29:31.190059 27547 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83d3897dd1874a22bb7551e08ad6b4af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "83d3897dd1874a22bb7551e08ad6b4af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83d3897dd1874a22bb7551e08ad6b4af" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 35997 } } }
I20260808 08:29:31.190166 27548 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83d3897dd1874a22bb7551e08ad6b4af [sys.catalog]: This master's current role is: LEADER
I20260808 08:29:31.190172 27547 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83d3897dd1874a22bb7551e08ad6b4af [sys.catalog]: This master's current role is: LEADER
I20260808 08:29:31.190512 27571 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:29:31.193317 27571 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:29:31.198027 27571 catalog_manager.cc:1347] Generated new cluster ID: d57acf6941c841faab3777e254f87128
I20260808 08:29:31.198092 27571 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:29:31.215605 27571 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:29:31.215912 27571 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:29:31.226037 27571 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 83d3897dd1874a22bb7551e08ad6b4af: Generated new TSK 0
I20260808 08:29:31.226822 27571 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 08:29:31.237079 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 27409
2026-08-08T08:29:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1537 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786177493204855-23471-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 08:29:31 dist-test-slave-f7th krb5kdc[27584](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 08:29:31 dist-test-slave-f7th krb5kdc[27584](info): set up 2 sockets
Aug 08 08:29:31 dist-test-slave-f7th krb5kdc[27584](info): commencing operation
krb5kdc: starting...
W20260808 08:29:32.409889 23471 mini_kdc.cc:121] Time spent starting KDC: real 1.147s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Aug 08 08:29:32 dist-test-slave-f7th krb5kdc[27584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177772, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T08:29:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:29:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.235.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
I20260808 08:29:32.529361 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:41025
--webserver_interface=127.22.235.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:45773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:41025
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:29:32.640493 27608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:29:32.640781 27608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:29:32.640843 27608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:29:32.645248 27608 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:29:32.645321 27608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:29:32.645350 27608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:29:32.645372 27608 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:29:32.645402 27608 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:29:32.650861 27608 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:45773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:41025
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:41025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.235.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27608
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:29:32.651963 27608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:29:32.653039 27608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 08:29:32.661495 27608 server_base.cc:1034] running on GCE node
W20260808 08:29:32.661443 27619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:32.661468 27618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:32.661443 27621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:32.662333 27608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:29:32.663419 27608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:29:32.664601 27608 hybrid_clock.cc:648] HybridClock initialized: now 1786177772664589 us; error 24 us; skew 500 ppm
Aug 08 08:29:32 dist-test-slave-f7th krb5kdc[27584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177772, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:29:32.667236 27608 init.cc:377] Logged in from keytab as kudu/127.22.235.254@KRBTEST.COM (short username kudu)
I20260808 08:29:32.668114 27608 webserver.cc:460] Webserver started at http://127.22.235.254:33757/ using document root <none> and password file <none>
I20260808 08:29:32.668687 27608 fs_manager.cc:359] Metadata directory not provided
I20260808 08:29:32.668761 27608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:29:32.668967 27608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:29:32.671016 27608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "3202515d918b4045a6745cb5165f559f"
format_stamp: "Formatted at 2026-08-08 08:29:32 on dist-test-slave-f7th"
server_key: "1fd97e0cc4cfaa65f364d96e4e2df061"
server_key_iv: "b1d7b653c245b962077983e509df924f"
server_key_version: "encryptionkey@0"
I20260808 08:29:32.671538 27608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "3202515d918b4045a6745cb5165f559f"
format_stamp: "Formatted at 2026-08-08 08:29:32 on dist-test-slave-f7th"
server_key: "1fd97e0cc4cfaa65f364d96e4e2df061"
server_key_iv: "b1d7b653c245b962077983e509df924f"
server_key_version: "encryptionkey@0"
I20260808 08:29:32.675231 27608 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260808 08:29:32.677600 27636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:32.678773 27608 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:29:32.678891 27608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "3202515d918b4045a6745cb5165f559f"
format_stamp: "Formatted at 2026-08-08 08:29:32 on dist-test-slave-f7th"
server_key: "1fd97e0cc4cfaa65f364d96e4e2df061"
server_key_iv: "b1d7b653c245b962077983e509df924f"
server_key_version: "encryptionkey@0"
I20260808 08:29:32.678984 27608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:29:32.694599 27608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:29:32.697417 27608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:29:32.697602 27608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:29:32.704456 27608 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.254:41025
I20260808 08:29:32.704490 27730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.254:41025 every 8 connection(s)
I20260808 08:29:32.705448 27608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:29:32.708318 27731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:29:32.713989 27731 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3202515d918b4045a6745cb5165f559f: Bootstrap starting.
I20260808 08:29:32.714900 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 27608
I20260808 08:29:32.715050 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:29:32.715346 23471 external_mini_cluster.cc:1428] Setting key 35f35426eee5804fd94ef3446407da4b
I20260808 08:29:32.716974 27731 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3202515d918b4045a6745cb5165f559f: Neither blocks nor log segments found. Creating new log.
I20260808 08:29:32.717729 27731 log.cc:824] T 00000000000000000000000000000000 P 3202515d918b4045a6745cb5165f559f: Log is configured to *not* fsync() on all Append() calls
I20260808 08:29:32.719856 27731 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3202515d918b4045a6745cb5165f559f: No bootstrap required, opened a new log
Aug 08 08:29:32 dist-test-slave-f7th krb5kdc[27584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177772, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:29:32.724203 27731 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3202515d918b4045a6745cb5165f559f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3202515d918b4045a6745cb5165f559f" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 41025 } }
I20260808 08:29:32.724439 27731 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3202515d918b4045a6745cb5165f559f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:29:32.724506 27731 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3202515d918b4045a6745cb5165f559f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3202515d918b4045a6745cb5165f559f, State: Initialized, Role: FOLLOWER
I20260808 08:29:32.724972 27731 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3202515d918b4045a6745cb5165f559f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3202515d918b4045a6745cb5165f559f" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 41025 } }
I20260808 08:29:32.725119 27731 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3202515d918b4045a6745cb5165f559f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:29:32.725186 27731 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3202515d918b4045a6745cb5165f559f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:29:32.725291 27731 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3202515d918b4045a6745cb5165f559f [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:29:32.726275 27731 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3202515d918b4045a6745cb5165f559f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3202515d918b4045a6745cb5165f559f" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 41025 } }
I20260808 08:29:32.726617 27731 leader_election.cc:302] T 00000000000000000000000000000000 P 3202515d918b4045a6745cb5165f559f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3202515d918b4045a6745cb5165f559f; no voters:
I20260808 08:29:32.726940 27731 leader_election.cc:288] T 00000000000000000000000000000000 P 3202515d918b4045a6745cb5165f559f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:29:32.727035 27737 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3202515d918b4045a6745cb5165f559f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:29:32.727252 27737 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3202515d918b4045a6745cb5165f559f [term 1 LEADER]: Becoming Leader. State: Replica: 3202515d918b4045a6745cb5165f559f, State: Running, Role: LEADER
I20260808 08:29:32.727612 27737 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3202515d918b4045a6745cb5165f559f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3202515d918b4045a6745cb5165f559f" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 41025 } }
I20260808 08:29:32.728331 27731 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3202515d918b4045a6745cb5165f559f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:29:32.729230 27739 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3202515d918b4045a6745cb5165f559f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3202515d918b4045a6745cb5165f559f. Latest consensus state: current_term: 1 leader_uuid: "3202515d918b4045a6745cb5165f559f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3202515d918b4045a6745cb5165f559f" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 41025 } } }
I20260808 08:29:32.729349 27739 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3202515d918b4045a6745cb5165f559f [sys.catalog]: This master's current role is: LEADER
I20260808 08:29:32.729719 27738 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3202515d918b4045a6745cb5165f559f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3202515d918b4045a6745cb5165f559f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3202515d918b4045a6745cb5165f559f" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 41025 } } }
I20260808 08:29:32.729774 27734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:32.716473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60407 (local address 127.22.235.254:41025)
0808 08:29:32.716900 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:32.716917 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:32.716953 (+ 36us) server_negotiation.cc:407] Connection header received
0808 08:29:32.717623 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:32.717641 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:32.718009 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:32.718381 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:32.719194 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:32.720192 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:32.721356 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:32.721821 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:32.721936 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:32.724738 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:32.724744 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:32.724762 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:32.724798 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:32.726476 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:32.726935 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:32.726942 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:32.726948 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:32.727106 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:32.727430 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:32.727434 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:32.727435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:32.727922 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:32.728118 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:32.728406 (+ 288us) server_negotiation.cc:299] Negotiation successful
0808 08:29:32.728757 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":148,"threads_started":1}
I20260808 08:29:32.729869 27738 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3202515d918b4045a6745cb5165f559f [sys.catalog]: This master's current role is: LEADER
I20260808 08:29:32.733063 23471 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260808 08:29:32.733616 27758 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3202515d918b4045a6745cb5165f559f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 08:29:32.733695 27758 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 08:29:32.733771 27750 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:29:32.734680 27750 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:29:32.740702 27750 catalog_manager.cc:1347] Generated new cluster ID: 8f5a0f4f80f641f8895a696037a5737c
I20260808 08:29:32.740764 27750 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:29:32.745121 27734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:32.737053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39111 (local address 127.22.235.254:41025)
0808 08:29:32.737152 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:32.737156 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:32.737344 (+ 188us) server_negotiation.cc:407] Connection header received
0808 08:29:32.737544 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:32.737547 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:32.737597 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:32.737682 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:32.738469 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:32.739246 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:32.740379 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:32.740769 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:32.740911 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:32.742176 (+ 1265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:32.742180 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:32.742182 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:32.742208 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:32.743517 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:32.743926 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:32.743930 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:32.743931 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:32.744012 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:32.744275 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:32.744279 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:32.744281 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:32.744471 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:32.744586 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:32.744865 (+ 279us) server_negotiation.cc:299] Negotiation successful
0808 08:29:32.744976 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 08:29:32.781524 27750 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:29:32.782841 27750 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:29:32.788398 27750 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3202515d918b4045a6745cb5165f559f: Generated new TSK 0
I20260808 08:29:32.789270 27750 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 08:29:32.797407 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 27608
2026-08-08T08:29:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1571 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786177493204855-23471-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 08:29:32 dist-test-slave-f7th krb5kdc[27774](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 08:29:32 dist-test-slave-f7th krb5kdc[27774](info): set up 2 sockets
Aug 08 08:29:32 dist-test-slave-f7th krb5kdc[27774](info): commencing operation
krb5kdc: starting...
W20260808 08:29:33.984540 23471 mini_kdc.cc:121] Time spent starting KDC: real 1.150s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Aug 08 08:29:34 dist-test-slave-f7th krb5kdc[27774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T08:29:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:29:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.235.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
I20260808 08:29:34.115388 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:46637
--webserver_interface=127.22.235.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:45391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:46637
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:29:34.229132 27795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:29:34.229408 27795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:29:34.229467 27795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:29:34.233560 27795 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:29:34.233628 27795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:29:34.233656 27795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:29:34.233680 27795 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:29:34.233709 27795 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:29:34.238981 27795 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:45391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:46637
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:46637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.235.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:29:34.240056 27795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:29:34.241168 27795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 08:29:34.250017 27795 server_base.cc:1034] running on GCE node
W20260808 08:29:34.250160 27806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:34.250186 27811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:34.250186 27804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:34.250952 27795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:29:34.252120 27795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:29:34.253294 27795 hybrid_clock.cc:648] HybridClock initialized: now 1786177774253265 us; error 41 us; skew 500 ppm
Aug 08 08:29:34 dist-test-slave-f7th krb5kdc[27774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177774, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:29:34.256069 27795 init.cc:377] Logged in from keytab as kudu/127.22.235.254@KRBTEST.COM (short username kudu)
I20260808 08:29:34.257062 27795 webserver.cc:460] Webserver started at http://127.22.235.254:38985/ using document root <none> and password file <none>
I20260808 08:29:34.257655 27795 fs_manager.cc:359] Metadata directory not provided
I20260808 08:29:34.257730 27795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:29:34.257948 27795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:29:34.260052 27795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "6d97344eddce468098cd0f26e6576f85"
format_stamp: "Formatted at 2026-08-08 08:29:34 on dist-test-slave-f7th"
server_key: "1af3c71049c26bbcfad4cdf1d0f37b6a"
server_key_iv: "b463587004580c6f226dd61e6c22dbdc"
server_key_version: "encryptionkey@0"
I20260808 08:29:34.260641 27795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "6d97344eddce468098cd0f26e6576f85"
format_stamp: "Formatted at 2026-08-08 08:29:34 on dist-test-slave-f7th"
server_key: "1af3c71049c26bbcfad4cdf1d0f37b6a"
server_key_iv: "b463587004580c6f226dd61e6c22dbdc"
server_key_version: "encryptionkey@0"
I20260808 08:29:34.264564 27795 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 08:29:34.267228 27821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:34.268589 27795 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 08:29:34.268748 27795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "6d97344eddce468098cd0f26e6576f85"
format_stamp: "Formatted at 2026-08-08 08:29:34 on dist-test-slave-f7th"
server_key: "1af3c71049c26bbcfad4cdf1d0f37b6a"
server_key_iv: "b463587004580c6f226dd61e6c22dbdc"
server_key_version: "encryptionkey@0"
I20260808 08:29:34.268854 27795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:29:34.282281 27795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:29:34.285362 27795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:29:34.285533 27795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:29:34.292814 27795 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.254:46637
I20260808 08:29:34.292852 27917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.254:46637 every 8 connection(s)
I20260808 08:29:34.293839 27795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:29:34.296885 27919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:29:34.302528 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 27795
I20260808 08:29:34.302693 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:29:34.302922 27919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d97344eddce468098cd0f26e6576f85: Bootstrap starting.
I20260808 08:29:34.303023 23471 external_mini_cluster.cc:1428] Setting key 30d9ed3a63e84196d0fee7dbfad95140
I20260808 08:29:34.305658 27919 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d97344eddce468098cd0f26e6576f85: Neither blocks nor log segments found. Creating new log.
I20260808 08:29:34.306807 27919 log.cc:824] T 00000000000000000000000000000000 P 6d97344eddce468098cd0f26e6576f85: Log is configured to *not* fsync() on all Append() calls
I20260808 08:29:34.309167 27919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d97344eddce468098cd0f26e6576f85: No bootstrap required, opened a new log
I20260808 08:29:34.312227 27922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:34.304409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46525 (local address 127.22.235.254:46637)
0808 08:29:34.304857 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:34.304875 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:34.304915 (+ 40us) server_negotiation.cc:407] Connection header received
0808 08:29:34.305599 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:34.305617 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:34.306022 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:34.306317 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:34.306765 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:34.307791 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:34.309003 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:34.309506 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:34.309627 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:34.310487 (+ 860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:34.310508 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:34.310522 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:29:34.310938 (+ 416us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:34.310996 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:29:34.311020 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:34.311099 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:34.311258 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 08:29:34.311457 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":138,"threads_started":1}
I20260808 08:29:34.313239 27919 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d97344eddce468098cd0f26e6576f85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d97344eddce468098cd0f26e6576f85" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46637 } }
I20260808 08:29:34.313467 27919 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d97344eddce468098cd0f26e6576f85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:29:34.313536 27919 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d97344eddce468098cd0f26e6576f85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d97344eddce468098cd0f26e6576f85, State: Initialized, Role: FOLLOWER
I20260808 08:29:34.313984 27919 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d97344eddce468098cd0f26e6576f85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d97344eddce468098cd0f26e6576f85" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46637 } }
I20260808 08:29:34.314124 27919 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d97344eddce468098cd0f26e6576f85 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:29:34.314225 27919 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d97344eddce468098cd0f26e6576f85 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:29:34.314322 27919 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d97344eddce468098cd0f26e6576f85 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:29:34.315006 23471 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 08:29:34.315287 27919 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d97344eddce468098cd0f26e6576f85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d97344eddce468098cd0f26e6576f85" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46637 } }
I20260808 08:29:34.315618 27919 leader_election.cc:302] T 00000000000000000000000000000000 P 6d97344eddce468098cd0f26e6576f85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d97344eddce468098cd0f26e6576f85; no voters:
I20260808 08:29:34.315910 27919 leader_election.cc:288] T 00000000000000000000000000000000 P 6d97344eddce468098cd0f26e6576f85 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:29:34.316033 27927 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d97344eddce468098cd0f26e6576f85 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:29:34.316246 27927 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d97344eddce468098cd0f26e6576f85 [term 1 LEADER]: Becoming Leader. State: Replica: 6d97344eddce468098cd0f26e6576f85, State: Running, Role: LEADER
I20260808 08:29:34.316676 27927 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d97344eddce468098cd0f26e6576f85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d97344eddce468098cd0f26e6576f85" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46637 } }
I20260808 08:29:34.317348 27919 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d97344eddce468098cd0f26e6576f85 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:29:34.318169 27931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d97344eddce468098cd0f26e6576f85 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d97344eddce468098cd0f26e6576f85. Latest consensus state: current_term: 1 leader_uuid: "6d97344eddce468098cd0f26e6576f85" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d97344eddce468098cd0f26e6576f85" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46637 } } }
I20260808 08:29:34.318392 27931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d97344eddce468098cd0f26e6576f85 [sys.catalog]: This master's current role is: LEADER
I20260808 08:29:34.318442 27929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d97344eddce468098cd0f26e6576f85 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d97344eddce468098cd0f26e6576f85" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d97344eddce468098cd0f26e6576f85" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 46637 } } }
I20260808 08:29:34.318518 27929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d97344eddce468098cd0f26e6576f85 [sys.catalog]: This master's current role is: LEADER
I20260808 08:29:34.318912 27948 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:29:34.322789 27948 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:29:34.323379 27922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:34.317234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56579 (local address 127.22.235.254:46637)
0808 08:29:34.317342 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:34.317346 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:34.317785 (+ 439us) server_negotiation.cc:407] Connection header received
0808 08:29:34.317938 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:34.317942 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:34.317991 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:34.318045 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:34.319014 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:34.319748 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:34.320913 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:34.321427 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:34.321605 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:34.322519 (+ 914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:34.322524 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:34.322528 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:29:34.322543 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:34.322593 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:29:34.322610 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:34.322740 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:34.323172 (+ 432us) server_negotiation.cc:299] Negotiation successful
0808 08:29:34.323230 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 08:29:34.329049 27948 catalog_manager.cc:1347] Generated new cluster ID: a48e909318404398a625599812bb23d1
I20260808 08:29:34.329116 27948 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:29:34.342163 27948 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:29:34.343401 27948 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:29:34.352075 27948 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d97344eddce468098cd0f26e6576f85: Generated new TSK 0
I20260808 08:29:34.352921 27948 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 08:29:34.356920 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 27795
2026-08-08T08:29:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1559 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786177493204855-23471-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 08:29:34 dist-test-slave-f7th krb5kdc[27967](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 08:29:34 dist-test-slave-f7th krb5kdc[27967](info): set up 2 sockets
Aug 08 08:29:34 dist-test-slave-f7th krb5kdc[27967](info): commencing operation
krb5kdc: starting...
W20260808 08:29:35.557797 23471 mini_kdc.cc:121] Time spent starting KDC: real 1.164s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Aug 08 08:29:35 dist-test-slave-f7th krb5kdc[27967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177775, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T08:29:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:29:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.235.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
I20260808 08:29:35.679452 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:45369
--webserver_interface=127.22.235.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:40193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:45369
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:29:35.793916 27987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:29:35.794203 27987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:29:35.794245 27987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:29:35.798269 27987 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:29:35.798331 27987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:29:35.798350 27987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:29:35.798367 27987 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:29:35.798380 27987 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:29:35.803680 27987 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:40193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:45369
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:45369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.235.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:29:35.804836 27987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:29:35.805920 27987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:29:35.814618 28010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:35.814652 28005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:35.814816 27987 server_base.cc:1034] running on GCE node
W20260808 08:29:35.814653 28006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:35.815538 27987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:29:35.816706 27987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:29:35.817874 27987 hybrid_clock.cc:648] HybridClock initialized: now 1786177775817838 us; error 49 us; skew 500 ppm
Aug 08 08:29:35 dist-test-slave-f7th krb5kdc[27967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177775, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:29:35.820677 27987 init.cc:377] Logged in from keytab as kudu/127.22.235.254@KRBTEST.COM (short username kudu)
I20260808 08:29:35.821633 27987 webserver.cc:460] Webserver started at http://127.22.235.254:39881/ using document root <none> and password file <none>
I20260808 08:29:35.822201 27987 fs_manager.cc:359] Metadata directory not provided
I20260808 08:29:35.822273 27987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:29:35.822508 27987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:29:35.824653 27987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "d47c220912c246ceb791dce141ad84d5"
format_stamp: "Formatted at 2026-08-08 08:29:35 on dist-test-slave-f7th"
server_key: "093da7b7202c4b4247b4d4c5ec14b8d0"
server_key_iv: "022c039463111aaa094b49de3551e00c"
server_key_version: "encryptionkey@0"
I20260808 08:29:35.825248 27987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "d47c220912c246ceb791dce141ad84d5"
format_stamp: "Formatted at 2026-08-08 08:29:35 on dist-test-slave-f7th"
server_key: "093da7b7202c4b4247b4d4c5ec14b8d0"
server_key_iv: "022c039463111aaa094b49de3551e00c"
server_key_version: "encryptionkey@0"
I20260808 08:29:35.829195 27987 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260808 08:29:35.831817 28018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:35.833145 27987 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:29:35.833276 27987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "d47c220912c246ceb791dce141ad84d5"
format_stamp: "Formatted at 2026-08-08 08:29:35 on dist-test-slave-f7th"
server_key: "093da7b7202c4b4247b4d4c5ec14b8d0"
server_key_iv: "022c039463111aaa094b49de3551e00c"
server_key_version: "encryptionkey@0"
I20260808 08:29:35.833380 27987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:29:35.838049 27987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:29:35.841270 27987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:29:35.841463 27987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:29:35.848883 27987 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.254:45369
I20260808 08:29:35.848927 28104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.254:45369 every 8 connection(s)
I20260808 08:29:35.849866 27987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:29:35.852893 28106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:29:35.855598 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 27987
I20260808 08:29:35.855731 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:29:35.856058 23471 external_mini_cluster.cc:1428] Setting key 23178d9d0a0661686d9efeefc63e92fa
I20260808 08:29:35.859472 28106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d47c220912c246ceb791dce141ad84d5: Bootstrap starting.
I20260808 08:29:35.861575 28113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:35.857607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53711 (local address 127.22.235.254:45369)
0808 08:29:35.858133 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:35.858148 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:35.858185 (+ 37us) server_negotiation.cc:407] Connection header received
0808 08:29:35.858859 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:35.858877 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:35.859179 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:35.859539 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:35.860351 (+ 812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:35.860357 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:35.860370 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:29:35.860380 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:35.860438 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:29:35.860458 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:35.860599 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:35.860706 (+ 107us) server_negotiation.cc:299] Negotiation successful
0808 08:29:35.860865 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":160,"threads_started":1}
I20260808 08:29:35.862142 28106 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d47c220912c246ceb791dce141ad84d5: Neither blocks nor log segments found. Creating new log.
I20260808 08:29:35.862953 28106 log.cc:824] T 00000000000000000000000000000000 P d47c220912c246ceb791dce141ad84d5: Log is configured to *not* fsync() on all Append() calls
I20260808 08:29:35.864401 23471 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 08:29:35.865144 28106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d47c220912c246ceb791dce141ad84d5: No bootstrap required, opened a new log
I20260808 08:29:35.868168 28113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:35.866526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48913 (local address 127.22.235.254:45369)
0808 08:29:35.866660 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:35.866663 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:35.866845 (+ 182us) server_negotiation.cc:407] Connection header received
0808 08:29:35.867012 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:35.867014 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:35.867047 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:35.867107 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:35.867803 (+ 696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:35.867806 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:35.867808 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 08:29:35.867817 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:35.867841 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 08:29:35.867851 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:35.867924 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:35.868031 (+ 107us) server_negotiation.cc:299] Negotiation successful
0808 08:29:35.868064 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 08:29:35.869639 28106 raft_consensus.cc:348] T 00000000000000000000000000000000 P d47c220912c246ceb791dce141ad84d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d47c220912c246ceb791dce141ad84d5" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 45369 } }
I20260808 08:29:35.869877 28106 raft_consensus.cc:374] T 00000000000000000000000000000000 P d47c220912c246ceb791dce141ad84d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 08:29:35.869937 28106 raft_consensus.cc:729] T 00000000000000000000000000000000 P d47c220912c246ceb791dce141ad84d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d47c220912c246ceb791dce141ad84d5, State: Initialized, Role: FOLLOWER
I20260808 08:29:35.870398 28106 consensus_queue.cc:260] T 00000000000000000000000000000000 P d47c220912c246ceb791dce141ad84d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d47c220912c246ceb791dce141ad84d5" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 45369 } }
I20260808 08:29:35.870589 28106 raft_consensus.cc:388] T 00000000000000000000000000000000 P d47c220912c246ceb791dce141ad84d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 08:29:35.870718 28106 raft_consensus.cc:482] T 00000000000000000000000000000000 P d47c220912c246ceb791dce141ad84d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 08:29:35.870837 28106 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d47c220912c246ceb791dce141ad84d5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:29:35.871896 28106 raft_consensus.cc:504] T 00000000000000000000000000000000 P d47c220912c246ceb791dce141ad84d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d47c220912c246ceb791dce141ad84d5" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 45369 } }
I20260808 08:29:35.872216 28106 leader_election.cc:302] T 00000000000000000000000000000000 P d47c220912c246ceb791dce141ad84d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d47c220912c246ceb791dce141ad84d5; no voters:
I20260808 08:29:35.872493 28106 leader_election.cc:288] T 00000000000000000000000000000000 P d47c220912c246ceb791dce141ad84d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 08:29:35.872607 28121 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d47c220912c246ceb791dce141ad84d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:29:35.872874 28121 raft_consensus.cc:686] T 00000000000000000000000000000000 P d47c220912c246ceb791dce141ad84d5 [term 1 LEADER]: Becoming Leader. State: Replica: d47c220912c246ceb791dce141ad84d5, State: Running, Role: LEADER
I20260808 08:29:35.873241 28121 consensus_queue.cc:237] T 00000000000000000000000000000000 P d47c220912c246ceb791dce141ad84d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d47c220912c246ceb791dce141ad84d5" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 45369 } }
I20260808 08:29:35.873780 28106 sys_catalog.cc:564] T 00000000000000000000000000000000 P d47c220912c246ceb791dce141ad84d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:29:35.874892 28124 sys_catalog.cc:455] T 00000000000000000000000000000000 P d47c220912c246ceb791dce141ad84d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d47c220912c246ceb791dce141ad84d5. Latest consensus state: current_term: 1 leader_uuid: "d47c220912c246ceb791dce141ad84d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d47c220912c246ceb791dce141ad84d5" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 45369 } } }
I20260808 08:29:35.875051 28124 sys_catalog.cc:458] T 00000000000000000000000000000000 P d47c220912c246ceb791dce141ad84d5 [sys.catalog]: This master's current role is: LEADER
I20260808 08:29:35.875388 28123 sys_catalog.cc:455] T 00000000000000000000000000000000 P d47c220912c246ceb791dce141ad84d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d47c220912c246ceb791dce141ad84d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d47c220912c246ceb791dce141ad84d5" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 45369 } } }
I20260808 08:29:35.875454 28138 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:29:35.875514 28123 sys_catalog.cc:458] T 00000000000000000000000000000000 P d47c220912c246ceb791dce141ad84d5 [sys.catalog]: This master's current role is: LEADER
I20260808 08:29:35.878553 28138 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:29:35.883898 28138 catalog_manager.cc:1347] Generated new cluster ID: 101d190ea5ad4bed90006b1a19911e2f
I20260808 08:29:35.883966 28138 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:29:35.897030 28138 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:29:35.897342 28138 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:29:35.909780 28138 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d47c220912c246ceb791dce141ad84d5: Generated new TSK 0
I20260808 08:29:35.910607 28138 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 08:29:35.925104 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 27987
2026-08-08T08:29:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1570 ms)
[----------] 18 tests from AuthTokenIssuingTest (27776 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 08:29:35 dist-test-slave-f7th krb5kdc[28157](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 08:29:35 dist-test-slave-f7th krb5kdc[28157](info): set up 2 sockets
Aug 08 08:29:35 dist-test-slave-f7th krb5kdc[28157](info): commencing operation
krb5kdc: starting...
W20260808 08:29:37.102880 23471 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Aug 08 08:29:37 dist-test-slave-f7th krb5kdc[28157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177777, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T08:29:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:29:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.235.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
I20260808 08:29:37.218977 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:38009
--webserver_interface=127.22.235.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:41135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:38009,127.22.235.253:46613,127.22.235.252:39223
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:29:37.331003 28183 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 08:29:37.331269 28183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:29:37.331292 28183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:29:37.331326 28183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:29:37.335203 28183 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:29:37.335255 28183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:29:37.335273 28183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:29:37.335292 28183 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:29:37.335307 28183 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:29:37.340396 28183 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:41135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:38009,127.22.235.253:46613,127.22.235.252:39223
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:38009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.235.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28183
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:29:37.341387 28183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:29:37.342348 28183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:29:37.350689 28189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:37.350809 28183 server_base.cc:1034] running on GCE node
W20260808 08:29:37.350689 28190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:37.350703 28195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:37.351459 28183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:29:37.352492 28183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:29:37.353727 28183 hybrid_clock.cc:648] HybridClock initialized: now 1786177777353708 us; error 30 us; skew 500 ppm
Aug 08 08:29:37 dist-test-slave-f7th krb5kdc[28157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177777, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:29:37.359706 28183 init.cc:377] Logged in from keytab as kudu/127.22.235.254@KRBTEST.COM (short username kudu)
I20260808 08:29:37.360586 28183 webserver.cc:460] Webserver started at http://127.22.235.254:46747/ using document root <none> and password file <none>
I20260808 08:29:37.361152 28183 fs_manager.cc:359] Metadata directory not provided
I20260808 08:29:37.361213 28183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:29:37.361409 28183 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:29:37.363395 28183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "c1094bde4f7a4dfa8243cded9d788084"
format_stamp: "Formatted at 2026-08-08 08:29:37 on dist-test-slave-f7th"
server_key: "9f4b18c5bdd368558d34e039f41c968f"
server_key_iv: "5e96da278e57d43dcbb565fcaea3d6c7"
server_key_version: "encryptionkey@0"
I20260808 08:29:37.363952 28183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "c1094bde4f7a4dfa8243cded9d788084"
format_stamp: "Formatted at 2026-08-08 08:29:37 on dist-test-slave-f7th"
server_key: "9f4b18c5bdd368558d34e039f41c968f"
server_key_iv: "5e96da278e57d43dcbb565fcaea3d6c7"
server_key_version: "encryptionkey@0"
I20260808 08:29:37.367621 28183 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 08:29:37.369995 28205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:37.371114 28183 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 08:29:37.371227 28183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "c1094bde4f7a4dfa8243cded9d788084"
format_stamp: "Formatted at 2026-08-08 08:29:37 on dist-test-slave-f7th"
server_key: "9f4b18c5bdd368558d34e039f41c968f"
server_key_iv: "5e96da278e57d43dcbb565fcaea3d6c7"
server_key_version: "encryptionkey@0"
I20260808 08:29:37.371315 28183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:29:37.394089 28183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:29:37.396798 28183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:29:37.396979 28183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:29:37.403556 28183 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.254:38009
I20260808 08:29:37.403906 28301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.254:38009 every 8 connection(s)
I20260808 08:29:37.404523 28183 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:29:37.407330 28302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:29:37.411041 28302 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38009 } has no permanent_uuid. Determining permanent_uuid...
I20260808 08:29:37.414460 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 28183
I20260808 08:29:37.414592 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:29:37.414911 23471 external_mini_cluster.cc:1428] Setting key b56132ef97f9427fa71eca13de36bca5
Aug 08 08:29:37 dist-test-slave-f7th krb5kdc[28157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177777, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.235.253@KRBTEST.COM; defaulting to no policy
I20260808 08:29:37.426422 28306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:37.413935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44670 (local address 127.22.235.254:38009)
0808 08:29:37.414191 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:37.414199 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:37.414234 (+ 35us) server_negotiation.cc:407] Connection header received
0808 08:29:37.415068 (+ 834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:37.415073 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:37.415251 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:37.415394 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:37.416414 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:37.417613 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:37.418636 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:37.419153 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:37.419305 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:37.421786 (+ 2481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:37.421795 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:37.421810 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:37.421837 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:37.423615 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:37.424067 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:37.424074 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:37.424076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:37.424211 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:37.424509 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:37.424512 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:37.424513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:37.424915 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:37.425038 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:37.425414 (+ 376us) server_negotiation.cc:299] Negotiation successful
0808 08:29:37.425712 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":136,"server-negotiator.queue_time_us":156,"thread_start_us":70,"threads_started":1}
I20260808 08:29:37.426442 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:37.413208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:38009 (local address 127.0.0.1:44670)
0808 08:29:37.413687 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:37.413729 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:37.414689 (+ 960us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:37.415458 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:37.415464 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:37.415929 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:29:37.416267 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:37.416285 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:37.417743 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:37.417746 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:37.418515 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:37.418525 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:37.419300 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:37.419305 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:37.419399 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:37.419955 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:29:37.419990 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:29:37.421631 (+ 1641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:29:37.423718 (+ 2087us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:29:37.423727 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:29:37.423735 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:29:37.423951 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:29:37.424299 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:29:37.424302 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:29:37.424303 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:29:37.424417 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:29:37.425035 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:29:37.425042 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:29:37.425200 (+ 158us) client_negotiation.cc:769] Sending connection context
0808 08:29:37.425346 (+ 146us) client_negotiation.cc:240] Negotiation successful
0808 08:29:37.425653 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":3,"thread_start_us":123,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.253@KRBTEST.COM" created.
I20260808 08:29:37.429093 28302 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46613 } has no permanent_uuid. Determining permanent_uuid...
I20260808 08:29:37.429761 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:37.429545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.253:46613 (local address 127.0.0.1:44850)
0808 08:29:37.429644 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:37.429709 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.253:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 08:29:37.429939 28212 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.235.253:46613: connect: Connection refused (error 111)
I20260808 08:29:37.430729 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:37.430543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.253:46613 (local address 127.0.0.1:44862)
0808 08:29:37.430641 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:37.430682 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.253:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 08:29:37.430874 28302 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.235.253:46613: Network error: Client connection negotiation failed: client connection to 127.22.235.253:46613: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.253.keytab.
Entry for principal kudu/127.22.235.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.253.keytab.
I20260808 08:29:37.442761 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.253:46613
--webserver_interface=127.22.235.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.253.keytab
--principal=kudu/127.22.235.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:41135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:38009,127.22.235.253:46613,127.22.235.252:39223
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 08:29:37.480103 28302 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46613 } attempt: 1
I20260808 08:29:37.480960 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:37.480677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.253:46613 (local address 127.0.0.1:44878)
0808 08:29:37.480818 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:37.480880 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.253:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 08:29:37.481655 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:37.481433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.253:46613 (local address 127.0.0.1:44888)
0808 08:29:37.481553 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:37.481598 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.253:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 08:29:37.481773 28302 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.235.253:46613: Network error: Client connection negotiation failed: client connection to 127.22.235.253:46613: connect: Connection refused (error 111)
I20260808 08:29:37.549916 28302 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46613 } attempt: 2
I20260808 08:29:37.550721 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:37.550442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.253:46613 (local address 127.0.0.1:44894)
0808 08:29:37.550586 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:37.550645 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.253:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 08:29:37.551390 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:37.551185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.253:46613 (local address 127.0.0.1:44896)
0808 08:29:37.551287 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:37.551331 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.253:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 08:29:37.551538 28302 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.235.253:46613: Network error: Client connection negotiation failed: client connection to 127.22.235.253:46613: connect: Connection refused (error 111)
W20260808 08:29:37.552404 28311 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 08:29:37.552687 28311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:29:37.552728 28311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:29:37.552791 28311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:29:37.556749 28311 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:29:37.556799 28311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:29:37.556824 28311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:29:37.556855 28311 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:29:37.556883 28311 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:29:37.561980 28311 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:41135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:38009,127.22.235.253:46613,127.22.235.252:39223
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.253.keytab
--principal=kudu/127.22.235.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.253:46613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.235.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:29:37.562990 28311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:29:37.564006 28311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:29:37.572023 28327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:37.572038 28321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:37.572016 28320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:37.572422 28311 server_base.cc:1034] running on GCE node
I20260808 08:29:37.572979 28311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:29:37.574008 28311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:29:37.575167 28311 hybrid_clock.cc:648] HybridClock initialized: now 1786177777575151 us; error 26 us; skew 500 ppm
Aug 08 08:29:37 dist-test-slave-f7th krb5kdc[28157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177777, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:29:37.577770 28311 init.cc:377] Logged in from keytab as kudu/127.22.235.253@KRBTEST.COM (short username kudu)
I20260808 08:29:37.578640 28311 webserver.cc:460] Webserver started at http://127.22.235.253:39567/ using document root <none> and password file <none>
I20260808 08:29:37.579123 28311 fs_manager.cc:359] Metadata directory not provided
I20260808 08:29:37.579177 28311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:29:37.579337 28311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:29:37.581270 28311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-1/data/instance:
uuid: "7a45ebce72e349fd81c0edc43036eb41"
format_stamp: "Formatted at 2026-08-08 08:29:37 on dist-test-slave-f7th"
server_key: "ce2506f98f7a395177b24d522b968d9d"
server_key_iv: "196a8853812da3870e7b539afcbbd1e3"
server_key_version: "encryptionkey@0"
I20260808 08:29:37.581753 28311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-1/wal/instance:
uuid: "7a45ebce72e349fd81c0edc43036eb41"
format_stamp: "Formatted at 2026-08-08 08:29:37 on dist-test-slave-f7th"
server_key: "ce2506f98f7a395177b24d522b968d9d"
server_key_iv: "196a8853812da3870e7b539afcbbd1e3"
server_key_version: "encryptionkey@0"
I20260808 08:29:37.585140 28311 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 08:29:37.587435 28336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:37.588665 28311 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:29:37.588769 28311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-1/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-1/wal
uuid: "7a45ebce72e349fd81c0edc43036eb41"
format_stamp: "Formatted at 2026-08-08 08:29:37 on dist-test-slave-f7th"
server_key: "ce2506f98f7a395177b24d522b968d9d"
server_key_iv: "196a8853812da3870e7b539afcbbd1e3"
server_key_version: "encryptionkey@0"
I20260808 08:29:37.588852 28311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:29:37.616158 28311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:29:37.618916 28311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:29:37.619068 28311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:29:37.625820 28311 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.253:46613
I20260808 08:29:37.626073 28420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.253:46613 every 8 connection(s)
I20260808 08:29:37.626761 28311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-1/data/info.pb
I20260808 08:29:37.628753 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 28311
I20260808 08:29:37.628862 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-1/wal/instance
I20260808 08:29:37.629137 23471 external_mini_cluster.cc:1428] Setting key e40f2cd3a550137b5d98677801bca7b7
I20260808 08:29:37.629684 28421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:29:37.633414 28421 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38009 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.22.235.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.252@KRBTEST.COM" created.
Aug 08 08:29:37 dist-test-slave-f7th krb5kdc[28157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177777, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.253@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:29:37.642688 28302 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46613 } attempt: 3
I20260808 08:29:37.647291 28306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:37.635458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44676 (local address 127.22.235.254:38009)
0808 08:29:37.635620 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:37.635625 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:37.636455 (+ 830us) server_negotiation.cc:407] Connection header received
0808 08:29:37.637406 (+ 951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:37.637410 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:37.637480 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:37.637576 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:37.638717 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:37.639238 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:37.640382 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:37.640669 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:37.640768 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:37.643296 (+ 2528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:37.643301 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:37.643303 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:37.643333 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:37.644987 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:37.645585 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:37.645588 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:37.645590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:37.645674 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:37.646061 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:37.646064 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:37.646066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:37.646273 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:37.646364 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:37.647005 (+ 641us) server_negotiation.cc:299] Negotiation successful
0808 08:29:37.647130 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Aug 08 08:29:37 dist-test-slave-f7th krb5kdc[28157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177777, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for kudu/127.22.235.253@KRBTEST.COM
I20260808 08:29:37.648594 28424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:37.635796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:38009 (local address 127.0.0.1:44676)
0808 08:29:37.636261 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:37.636337 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:37.637189 (+ 852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:37.637755 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:37.637765 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:37.638091 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:29:37.638574 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:37.638588 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:37.639343 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:37.639346 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:37.640272 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:37.640282 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:37.640823 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:37.640828 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:37.641000 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:37.641527 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:29:37.641544 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:29:37.643106 (+ 1562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:29:37.645108 (+ 2002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:29:37.645114 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:29:37.645129 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:29:37.645484 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:29:37.645767 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:29:37.645770 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:29:37.645772 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:29:37.645955 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:29:37.646379 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:29:37.646386 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:29:37.646711 (+ 325us) client_negotiation.cc:769] Sending connection context
0808 08:29:37.646941 (+ 230us) client_negotiation.cc:240] Negotiation successful
0808 08:29:37.647261 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":113,"threads_started":1}
I20260808 08:29:37.650350 28421 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46613 } has no permanent_uuid. Determining permanent_uuid...
I20260808 08:29:37.652638 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:37.643102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.253:46613 (local address 127.0.0.1:44910)
0808 08:29:37.643235 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:37.643248 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:37.643326 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:37.643965 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:37.643968 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:37.644128 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:29:37.644717 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:37.644729 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:37.645820 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:37.645823 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:37.646502 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:37.646510 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:37.647106 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:37.647110 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:37.647202 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:37.647709 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:29:37.647720 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:29:37.648920 (+ 1200us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:29:37.650980 (+ 2060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:29:37.650983 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:29:37.650985 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:29:37.651225 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:29:37.651568 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:29:37.651572 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:29:37.651574 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:29:37.651653 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:29:37.652093 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:29:37.652096 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:29:37.652193 (+ 97us) client_negotiation.cc:769] Sending connection context
0808 08:29:37.652281 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 08:29:37.652467 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 08:29:37.652755 28429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:37.643219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44910 (local address 127.22.235.253:46613)
0808 08:29:37.643564 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:37.643578 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:37.643601 (+ 23us) server_negotiation.cc:407] Connection header received
0808 08:29:37.643641 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:37.643646 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:37.643826 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:37.643958 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:37.644862 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:37.645698 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:37.646597 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:37.646965 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:37.647401 (+ 436us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:37.649079 (+ 1678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:37.649086 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:37.649098 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:37.649133 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:37.650852 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:37.651367 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:37.651373 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:37.651375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:37.651475 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:37.651736 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:37.651739 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:37.651741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:37.651988 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:37.652106 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:37.652375 (+ 269us) server_negotiation.cc:299] Negotiation successful
0808 08:29:37.652566 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":231,"thread_start_us":147,"threads_started":1}
I20260808 08:29:37.654843 28302 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.235.252" port: 39223 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 08:29:37 dist-test-slave-f7th krb5kdc[28157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177777, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.253@KRBTEST.COM for kudu/127.22.235.253@KRBTEST.COM
I20260808 08:29:37.655481 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:37.655261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:39223 (local address 127.0.0.1:46172)
0808 08:29:37.655370 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:37.655423 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.252:39223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.252.keytab.
Entry for principal kudu/127.22.235.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.252.keytab.
I20260808 08:29:37.656772 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:37.656243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:39223 (local address 127.0.0.1:46174)
0808 08:29:37.656657 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:37.656710 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.252:39223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322}
W20260808 08:29:37.656920 28302 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.235.252:39223: Network error: Client connection negotiation failed: client connection to 127.22.235.252:39223: connect: Connection refused (error 111)
I20260808 08:29:37.657258 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.252:39223
--webserver_interface=127.22.235.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.252.keytab
--principal=kudu/127.22.235.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:41135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:38009,127.22.235.253:46613,127.22.235.252:39223
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 08:29:37.658604 28424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:37.650864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.253:46613 (local address 127.0.0.1:44912)
0808 08:29:37.650949 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:37.650962 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:37.651027 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:37.651379 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:37.651381 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:37.651528 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:29:37.651605 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:37.651610 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:37.652323 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:37.652326 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:37.653161 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:37.653173 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:37.653630 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:37.653634 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:37.653722 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:37.654100 (+ 378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:29:37.654114 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:29:37.655477 (+ 1363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:29:37.657287 (+ 1810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:29:37.657290 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:29:37.657292 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:29:37.657508 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:29:37.657749 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:29:37.657752 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:29:37.657754 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:29:37.657826 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:29:37.658148 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:29:37.658151 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:29:37.658228 (+ 77us) client_negotiation.cc:769] Sending connection context
0808 08:29:37.658308 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 08:29:37.658429 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24,"mutex_wait_us":66}
I20260808 08:29:37.658643 28430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:37.650925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44912 (local address 127.22.235.253:46613)
0808 08:29:37.651213 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:37.651219 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:37.651229 (+ 10us) server_negotiation.cc:407] Connection header received
0808 08:29:37.651253 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:37.651254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:37.651283 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:37.651353 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:37.651699 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:37.652231 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:37.653264 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:37.653529 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:37.653612 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:37.655633 (+ 2021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:37.655637 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:37.655639 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:37.655664 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:37.657191 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:37.657597 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:37.657600 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:37.657602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:37.657676 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:37.657898 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:37.657901 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:37.657903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:37.658056 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:37.658144 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:37.658360 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 08:29:37.658482 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":235,"thread_start_us":119,"threads_started":1}
I20260808 08:29:37.659374 28421 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.235.252" port: 39223 } has no permanent_uuid. Determining permanent_uuid...
I20260808 08:29:37.659943 28424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:37.659730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:39223 (local address 127.0.0.1:46188)
0808 08:29:37.659829 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:37.659895 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.252:39223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 08:29:37.660141 28337 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.235.252:39223: connect: Connection refused (error 111)
I20260808 08:29:37.660910 28424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:37.660733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:39223 (local address 127.0.0.1:46196)
0808 08:29:37.660823 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:37.660864 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.252:39223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 08:29:37.661037 28421 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.235.252:39223: Network error: Client connection negotiation failed: client connection to 127.22.235.252:39223: connect: Connection refused (error 111)
I20260808 08:29:37.688098 28302 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.235.252" port: 39223 } attempt: 1
I20260808 08:29:37.688997 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:37.688607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:39223 (local address 127.0.0.1:46200)
0808 08:29:37.688824 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:37.688896 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.252:39223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 08:29:37.689743 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:37.689505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:39223 (local address 127.0.0.1:46208)
0808 08:29:37.689620 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:37.689675 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.252:39223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 08:29:37.689873 28302 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.235.252:39223: Network error: Client connection negotiation failed: client connection to 127.22.235.252:39223: connect: Connection refused (error 111)
I20260808 08:29:37.710278 28421 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.235.252" port: 39223 } attempt: 1
I20260808 08:29:37.711020 28424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:37.710751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:39223 (local address 127.0.0.1:46214)
0808 08:29:37.710889 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:37.710947 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.252:39223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 08:29:37.711799 28424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:37.711563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:39223 (local address 127.0.0.1:46222)
0808 08:29:37.711694 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:37.711743 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.252:39223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260808 08:29:37.711930 28421 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.235.252:39223: Network error: Client connection negotiation failed: client connection to 127.22.235.252:39223: connect: Connection refused (error 111)
I20260808 08:29:37.765036 28302 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.235.252" port: 39223 } attempt: 2
I20260808 08:29:37.765910 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:37.765585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:39223 (local address 127.0.0.1:46230)
0808 08:29:37.765747 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:37.765815 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.252:39223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260808 08:29:37.766594 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:37.766376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:39223 (local address 127.0.0.1:46246)
0808 08:29:37.766480 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:37.766529 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.252:39223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 08:29:37.766710 28302 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.235.252:39223: Network error: Client connection negotiation failed: client connection to 127.22.235.252:39223: connect: Connection refused (error 111)
W20260808 08:29:37.769134 28433 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 08:29:37.769393 28433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:29:37.769434 28433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:29:37.769496 28433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:29:37.773387 28433 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:29:37.773437 28433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:29:37.773463 28433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:29:37.773495 28433 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:29:37.773518 28433 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:29:37.778717 28433 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:41135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:38009,127.22.235.253:46613,127.22.235.252:39223
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.252.keytab
--principal=kudu/127.22.235.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.252:39223
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.235.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:29:37.779691 28433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:29:37.780068 28421 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.235.252" port: 39223 } attempt: 2
I20260808 08:29:37.780779 28424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:37.780520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:39223 (local address 127.0.0.1:46258)
0808 08:29:37.780659 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:37.780710 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.252:39223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 08:29:37.780797 28433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 08:29:37.781406 28424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:37.781206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:39223 (local address 127.0.0.1:46266)
0808 08:29:37.781307 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:37.781351 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.252:39223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 08:29:37.781517 28421 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.235.252:39223: Network error: Client connection negotiation failed: client connection to 127.22.235.252:39223: connect: Connection refused (error 111)
W20260808 08:29:37.788967 28442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:37.788992 28443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:37.789005 28446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:37.789247 28433 server_base.cc:1034] running on GCE node
I20260808 08:29:37.789773 28433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:29:37.790802 28433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:29:37.791956 28433 hybrid_clock.cc:648] HybridClock initialized: now 1786177777791936 us; error 30 us; skew 500 ppm
Aug 08 08:29:37 dist-test-slave-f7th krb5kdc[28157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177777, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:29:37.794548 28433 init.cc:377] Logged in from keytab as kudu/127.22.235.252@KRBTEST.COM (short username kudu)
I20260808 08:29:37.795364 28433 webserver.cc:460] Webserver started at http://127.22.235.252:37437/ using document root <none> and password file <none>
I20260808 08:29:37.795885 28433 fs_manager.cc:359] Metadata directory not provided
I20260808 08:29:37.795953 28433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:29:37.796154 28433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:29:37.798141 28433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-2/data/instance:
uuid: "10fcd1293d6a4760a23b56da779e63f1"
format_stamp: "Formatted at 2026-08-08 08:29:37 on dist-test-slave-f7th"
server_key: "4809121989cde04b98da156c9917e446"
server_key_iv: "c74023ad8540db4461fee86176bea4ef"
server_key_version: "encryptionkey@0"
I20260808 08:29:37.798643 28433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-2/wal/instance:
uuid: "10fcd1293d6a4760a23b56da779e63f1"
format_stamp: "Formatted at 2026-08-08 08:29:37 on dist-test-slave-f7th"
server_key: "4809121989cde04b98da156c9917e446"
server_key_iv: "c74023ad8540db4461fee86176bea4ef"
server_key_version: "encryptionkey@0"
I20260808 08:29:37.802222 28433 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 08:29:37.804584 28453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:37.805783 28433 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:29:37.805898 28433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-2/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-2/wal
uuid: "10fcd1293d6a4760a23b56da779e63f1"
format_stamp: "Formatted at 2026-08-08 08:29:37 on dist-test-slave-f7th"
server_key: "4809121989cde04b98da156c9917e446"
server_key_iv: "c74023ad8540db4461fee86176bea4ef"
server_key_version: "encryptionkey@0"
I20260808 08:29:37.805987 28433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:29:37.828140 28433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:29:37.830880 28433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:29:37.831064 28433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:29:37.837769 28433 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.252:39223
I20260808 08:29:37.838020 28552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.252:39223 every 8 connection(s)
I20260808 08:29:37.838711 28433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-2/data/info.pb
I20260808 08:29:37.841452 28554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:29:37.843977 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 28433
I20260808 08:29:37.844111 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786177493204855-23471-0/minicluster-data/master-2/wal/instance
I20260808 08:29:37.844408 23471 external_mini_cluster.cc:1428] Setting key 62233833a3e7ca61b2f03f46b33dce6c
I20260808 08:29:37.845709 28554 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38009 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 08:29:37 dist-test-slave-f7th krb5kdc[28157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177777, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.252@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
Aug 08 08:29:37 dist-test-slave-f7th krb5kdc[28157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177777, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.235.252@KRBTEST.COM
I20260808 08:29:37.857348 28306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:37.847605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44692 (local address 127.22.235.254:38009)
0808 08:29:37.847755 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:37.847759 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:37.847985 (+ 226us) server_negotiation.cc:407] Connection header received
0808 08:29:37.848198 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:37.848201 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:37.848244 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:37.848292 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:37.849123 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:37.849857 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:37.850796 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:37.851016 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:37.851107 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:37.853688 (+ 2581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:37.853692 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:37.853694 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:37.853719 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:37.855149 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:37.855693 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:37.855696 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:37.855698 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:37.855790 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:37.856130 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:37.856133 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:37.856134 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:37.856348 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:37.856426 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:37.857062 (+ 636us) server_negotiation.cc:299] Negotiation successful
0808 08:29:37.857182 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 08:29:37.858171 28558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:37.847512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:38009 (local address 127.0.0.1:44692)
0808 08:29:37.847851 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:37.847899 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:37.848082 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:37.848444 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:37.848449 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:37.848817 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:29:37.848978 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:37.848989 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:37.849952 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:37.849954 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:37.850675 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:37.850684 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:37.851156 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:37.851160 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:37.851282 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:37.851969 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:29:37.851986 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:29:37.853498 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:29:37.855251 (+ 1753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:29:37.855258 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:29:37.855266 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:29:37.855600 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:29:37.855872 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:29:37.855875 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:29:37.855877 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:29:37.856046 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:29:37.856444 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:29:37.856451 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:29:37.856800 (+ 349us) client_negotiation.cc:769] Sending connection context
0808 08:29:37.856964 (+ 164us) client_negotiation.cc:240] Negotiation successful
0808 08:29:37.857292 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":68,"thread_start_us":81,"threads_started":1}
I20260808 08:29:37.858172 28557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:37.845873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46276 (local address 127.22.235.252:39223)
0808 08:29:37.846332 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:37.846352 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:37.846397 (+ 45us) server_negotiation.cc:407] Connection header received
0808 08:29:37.847107 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:37.847131 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:37.847467 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:37.847826 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:37.848687 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:37.849986 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:37.850797 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:37.851149 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:37.851286 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:37.854010 (+ 2724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:37.854016 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:37.854026 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:37.854047 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:37.855659 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:37.856104 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:37.856110 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:37.856112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:37.856203 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:37.856448 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:37.856451 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:37.856453 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:37.856808 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:37.856981 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:37.857220 (+ 239us) server_negotiation.cc:299] Negotiation successful
0808 08:29:37.857353 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":314,"spinlock_wait_cycles":7808,"thread_start_us":120,"threads_started":1}
I20260808 08:29:37.860014 28554 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46613 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 08:29:37 dist-test-slave-f7th krb5kdc[28157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177777, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.252@KRBTEST.COM for kudu/127.22.235.253@KRBTEST.COM
Aug 08 08:29:37 dist-test-slave-f7th krb5kdc[28157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177777, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:29:37.865840 28302 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.235.252" port: 39223 } attempt: 3
I20260808 08:29:37.868212 28558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:37.860492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.253:46613 (local address 127.0.0.1:44916)
0808 08:29:37.860668 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:37.860681 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:37.860749 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:37.860997 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:37.861000 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:37.861166 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:29:37.861260 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:37.861266 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:37.862094 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:37.862098 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:37.862658 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:37.862666 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:37.863144 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:37.863148 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:37.863235 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:37.863617 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:29:37.863627 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:29:37.864929 (+ 1302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:29:37.866684 (+ 1755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:29:37.866688 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:29:37.866690 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:29:37.866938 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:29:37.867254 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:29:37.867257 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:29:37.867259 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:29:37.867348 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:29:37.867756 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:29:37.867759 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:29:37.867842 (+ 83us) client_negotiation.cc:769] Sending connection context
0808 08:29:37.867927 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 08:29:37.868058 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92}
I20260808 08:29:37.868235 28430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:37.860609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44916 (local address 127.22.235.253:46613)
0808 08:29:37.860740 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:37.860744 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:37.860756 (+ 12us) server_negotiation.cc:407] Connection header received
0808 08:29:37.860838 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:37.860842 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:37.860892 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:37.860960 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:37.861388 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:37.861998 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:37.862763 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:37.863050 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:37.863124 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:37.865067 (+ 1943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:37.865071 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:37.865073 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:37.865101 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:37.866591 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:37.867049 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:37.867053 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:37.867054 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:37.867162 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:37.867480 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:37.867483 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:37.867485 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:37.867661 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:37.867745 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:37.867991 (+ 246us) server_negotiation.cc:299] Negotiation successful
0808 08:29:37.868102 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 08:29:37.869215 28554 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.235.252" port: 39223 } has no permanent_uuid. Determining permanent_uuid...
I20260808 08:29:37.869372 28306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:37.860564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44694 (local address 127.22.235.254:38009)
0808 08:29:37.860676 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:37.860680 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:37.860869 (+ 189us) server_negotiation.cc:407] Connection header received
0808 08:29:37.860969 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:37.860971 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:37.861016 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:37.861090 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:37.861672 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:37.862170 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:37.862895 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:37.863100 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:37.863196 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:37.865621 (+ 2425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:37.865623 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:37.865625 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:37.865646 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:37.867348 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:37.867787 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:37.867790 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:37.867792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:37.867870 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:37.868148 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:37.868151 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:37.868153 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:37.868345 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:37.868544 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:37.869132 (+ 588us) server_negotiation.cc:299] Negotiation successful
0808 08:29:37.869234 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
Aug 08 08:29:37 dist-test-slave-f7th krb5kdc[28157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177777, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for kudu/127.22.235.252@KRBTEST.COM
I20260808 08:29:37.872676 28421 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.235.252" port: 39223 } attempt: 3
I20260808 08:29:37.874518 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:37.866183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:39223 (local address 127.0.0.1:46282)
0808 08:29:37.866315 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:37.866330 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:37.866406 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:37.866853 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:37.866857 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:37.867145 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:29:37.867241 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:37.867248 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:37.868124 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:37.868127 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:37.868887 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:37.868898 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:37.869458 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:37.869461 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:37.869555 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:37.869951 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:29:37.869964 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:29:37.871237 (+ 1273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:29:37.873019 (+ 1782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:29:37.873022 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:29:37.873024 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:29:37.873251 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:29:37.873522 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:29:37.873525 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:29:37.873527 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:29:37.873614 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:29:37.874046 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:29:37.874048 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:29:37.874135 (+ 87us) client_negotiation.cc:769] Sending connection context
0808 08:29:37.874230 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 08:29:37.874348 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 08:29:37.875066 28557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:37.866560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46282 (local address 127.22.235.252:39223)
0808 08:29:37.866668 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:37.866672 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:37.866683 (+ 11us) server_negotiation.cc:407] Connection header received
0808 08:29:37.866711 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:37.866714 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:37.866749 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:37.866821 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:37.867329 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:37.868007 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:37.869023 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:37.869330 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:37.869738 (+ 408us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:37.871351 (+ 1613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:37.871355 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:37.871357 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:37.871383 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:37.872906 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:37.873340 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:37.873343 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:37.873345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:37.873420 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:37.873707 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:37.873710 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:37.873712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:37.873920 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:37.874003 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:37.874809 (+ 806us) server_negotiation.cc:299] Negotiation successful
0808 08:29:37.874917 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":48}
Aug 08 08:29:37 dist-test-slave-f7th krb5kdc[28157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177777, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.252@KRBTEST.COM for kudu/127.22.235.252@KRBTEST.COM
Aug 08 08:29:37 dist-test-slave-f7th krb5kdc[28157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177777, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.235.253@KRBTEST.COM
I20260808 08:29:37.878741 28558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:37.869658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:39223 (local address 127.0.0.1:46292)
0808 08:29:37.870052 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:37.870065 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:37.870124 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:37.870804 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:37.870807 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:37.870957 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:29:37.871034 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:37.871040 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:37.872331 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:37.872334 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:37.873114 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:37.873123 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:37.873727 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:37.873730 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:37.873837 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:37.874204 (+ 367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:29:37.874218 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:29:37.875522 (+ 1304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:29:37.877370 (+ 1848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:29:37.877372 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:29:37.877374 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:29:37.877602 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:29:37.877846 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:29:37.877849 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:29:37.877850 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:29:37.877922 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:29:37.878324 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:29:37.878326 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:29:37.878396 (+ 70us) client_negotiation.cc:769] Sending connection context
0808 08:29:37.878480 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 08:29:37.878602 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"mutex_wait_us":24}
I20260808 08:29:37.878755 28302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1094bde4f7a4dfa8243cded9d788084: Bootstrap starting.
I20260808 08:29:37.879170 28561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:37.869728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46292 (local address 127.22.235.252:39223)
0808 08:29:37.870550 (+ 822us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:37.870558 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:37.870571 (+ 13us) server_negotiation.cc:407] Connection header received
0808 08:29:37.870602 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:37.870604 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:37.870643 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:37.870716 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:37.871118 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:37.872220 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:37.873223 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:37.873553 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:37.873638 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:37.875635 (+ 1997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:37.875638 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:37.875641 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:37.875663 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:37.877263 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:37.877694 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:37.877697 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:37.877699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:37.877768 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:37.878002 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:37.878004 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:37.878006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:37.878214 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:37.878295 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:37.878924 (+ 629us) server_negotiation.cc:299] Negotiation successful
0808 08:29:37.879024 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":766,"thread_start_us":118,"threads_started":1}
Aug 08 08:29:37 dist-test-slave-f7th krb5kdc[28157](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177777, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.253@KRBTEST.COM for kudu/127.22.235.252@KRBTEST.COM
I20260808 08:29:37.881583 28302 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1094bde4f7a4dfa8243cded9d788084: Neither blocks nor log segments found. Creating new log.
I20260808 08:29:37.882323 28430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:37.870831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44920 (local address 127.22.235.253:46613)
0808 08:29:37.871386 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:37.871389 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:37.871398 (+ 9us) server_negotiation.cc:407] Connection header received
0808 08:29:37.871424 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:37.871426 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:37.871457 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:37.871540 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:37.872332 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:37.872917 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:37.873879 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:37.874434 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:37.874517 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:37.877813 (+ 3296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:37.877817 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:37.877820 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:37.877844 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:37.880139 (+ 2295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:37.880598 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:37.880601 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:37.880603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:37.880727 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:37.881047 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:37.881050 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:37.881052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:37.881256 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:37.881683 (+ 427us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:37.882079 (+ 396us) server_negotiation.cc:299] Negotiation successful
0808 08:29:37.882187 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":488}
I20260808 08:29:37.882544 28554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10fcd1293d6a4760a23b56da779e63f1: Bootstrap starting.
I20260808 08:29:37.882534 28302 log.cc:824] T 00000000000000000000000000000000 P c1094bde4f7a4dfa8243cded9d788084: Log is configured to *not* fsync() on all Append() calls
I20260808 08:29:37.884377 28424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:37.873045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:39223 (local address 127.0.0.1:46296)
0808 08:29:37.873279 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:37.873292 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:37.873358 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:37.873921 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:37.873924 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:37.874104 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:29:37.874193 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:37.874200 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:37.876298 (+ 2098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:37.876301 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:37.877089 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:37.877102 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:37.877766 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:37.877770 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:37.877854 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:37.878255 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:29:37.878269 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:29:37.880959 (+ 2690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:29:37.883009 (+ 2050us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:29:37.883012 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:29:37.883014 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:29:37.883194 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:29:37.883464 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:29:37.883467 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:29:37.883469 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:29:37.883557 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:29:37.883944 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:29:37.883947 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:29:37.884033 (+ 86us) client_negotiation.cc:769] Sending connection context
0808 08:29:37.884115 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 08:29:37.884233 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166}
I20260808 08:29:37.884418 28563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:37.873477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46296 (local address 127.22.235.252:39223)
0808 08:29:37.873706 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:37.873715 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:37.873728 (+ 13us) server_negotiation.cc:407] Connection header received
0808 08:29:37.873759 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:37.873762 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:37.873800 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:37.874296 (+ 496us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:37.874462 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:37.876168 (+ 1706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:37.877238 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:37.877625 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:37.878062 (+ 437us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:37.881117 (+ 3055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:37.881120 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:37.881123 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:37.881145 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:37.882872 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:37.883300 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:37.883303 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:37.883304 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:37.883389 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:37.883640 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:37.883643 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:37.883645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:37.883832 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:37.883948 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:37.884178 (+ 230us) server_negotiation.cc:299] Negotiation successful
0808 08:29:37.884273 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":87,"threads_started":1}
I20260808 08:29:37.885452 28554 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10fcd1293d6a4760a23b56da779e63f1: Neither blocks nor log segments found. Creating new log.
I20260808 08:29:37.886272 28554 log.cc:824] T 00000000000000000000000000000000 P 10fcd1293d6a4760a23b56da779e63f1: Log is configured to *not* fsync() on all Append() calls
I20260808 08:29:37.887290 28421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a45ebce72e349fd81c0edc43036eb41: Bootstrap starting.
I20260808 08:29:37.888398 28554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10fcd1293d6a4760a23b56da779e63f1: No bootstrap required, opened a new log
I20260808 08:29:37.889705 28421 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a45ebce72e349fd81c0edc43036eb41: Neither blocks nor log segments found. Creating new log.
I20260808 08:29:37.889740 28302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1094bde4f7a4dfa8243cded9d788084: No bootstrap required, opened a new log
I20260808 08:29:37.890492 28421 log.cc:824] T 00000000000000000000000000000000 P 7a45ebce72e349fd81c0edc43036eb41: Log is configured to *not* fsync() on all Append() calls
I20260808 08:29:37.891863 23471 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 08:29:37.892953 28554 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10fcd1293d6a4760a23b56da779e63f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1094bde4f7a4dfa8243cded9d788084" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38009 } } peers { permanent_uuid: "7a45ebce72e349fd81c0edc43036eb41" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46613 } } peers { permanent_uuid: "10fcd1293d6a4760a23b56da779e63f1" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 39223 } }
I20260808 08:29:37.892975 28302 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1094bde4f7a4dfa8243cded9d788084 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1094bde4f7a4dfa8243cded9d788084" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38009 } } peers { permanent_uuid: "7a45ebce72e349fd81c0edc43036eb41" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46613 } } peers { permanent_uuid: "10fcd1293d6a4760a23b56da779e63f1" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 39223 } }
I20260808 08:29:37.893209 28554 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10fcd1293d6a4760a23b56da779e63f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10fcd1293d6a4760a23b56da779e63f1, State: Initialized, Role: FOLLOWER
I20260808 08:29:37.893409 28252 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "c1094bde4f7a4dfa8243cded9d788084"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44694
I20260808 08:29:37.893545 28554 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10fcd1293d6a4760a23b56da779e63f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1094bde4f7a4dfa8243cded9d788084" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38009 } } peers { permanent_uuid: "7a45ebce72e349fd81c0edc43036eb41" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46613 } } peers { permanent_uuid: "10fcd1293d6a4760a23b56da779e63f1" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 39223 } }
I20260808 08:29:37.893649 28302 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1094bde4f7a4dfa8243cded9d788084 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1094bde4f7a4dfa8243cded9d788084, State: Initialized, Role: FOLLOWER
I20260808 08:29:37.894021 28302 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1094bde4f7a4dfa8243cded9d788084 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1094bde4f7a4dfa8243cded9d788084" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38009 } } peers { permanent_uuid: "7a45ebce72e349fd81c0edc43036eb41" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46613 } } peers { permanent_uuid: "10fcd1293d6a4760a23b56da779e63f1" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 39223 } }
I20260808 08:29:37.894096 28421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a45ebce72e349fd81c0edc43036eb41: No bootstrap required, opened a new log
I20260808 08:29:37.894541 28575 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1094bde4f7a4dfa8243cded9d788084 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1094bde4f7a4dfa8243cded9d788084" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38009 } } peers { permanent_uuid: "7a45ebce72e349fd81c0edc43036eb41" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46613 } } peers { permanent_uuid: "10fcd1293d6a4760a23b56da779e63f1" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 39223 } } }
I20260808 08:29:37.894852 28575 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1094bde4f7a4dfa8243cded9d788084 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 08:29:37.894891 28554 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10fcd1293d6a4760a23b56da779e63f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:29:37.896356 28302 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1094bde4f7a4dfa8243cded9d788084 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:29:37.896411 28252 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1094bde4f7a4dfa8243cded9d788084 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 08:29:37.896525 28252 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1094bde4f7a4dfa8243cded9d788084 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:29:37.897735 28252 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1094bde4f7a4dfa8243cded9d788084 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1094bde4f7a4dfa8243cded9d788084" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38009 } } peers { permanent_uuid: "7a45ebce72e349fd81c0edc43036eb41" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46613 } } peers { permanent_uuid: "10fcd1293d6a4760a23b56da779e63f1" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 39223 } }
I20260808 08:29:37.897888 28421 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a45ebce72e349fd81c0edc43036eb41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1094bde4f7a4dfa8243cded9d788084" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38009 } } peers { permanent_uuid: "7a45ebce72e349fd81c0edc43036eb41" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46613 } } peers { permanent_uuid: "10fcd1293d6a4760a23b56da779e63f1" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 39223 } }
I20260808 08:29:37.898092 28421 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a45ebce72e349fd81c0edc43036eb41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a45ebce72e349fd81c0edc43036eb41, State: Initialized, Role: FOLLOWER
I20260808 08:29:37.898373 28421 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a45ebce72e349fd81c0edc43036eb41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1094bde4f7a4dfa8243cded9d788084" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38009 } } peers { permanent_uuid: "7a45ebce72e349fd81c0edc43036eb41" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46613 } } peers { permanent_uuid: "10fcd1293d6a4760a23b56da779e63f1" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 39223 } }
I20260808 08:29:37.898669 28252 leader_election.cc:288] T 00000000000000000000000000000000 P c1094bde4f7a4dfa8243cded9d788084 [CANDIDATE]: Term 1 election: Requested vote from peers 7a45ebce72e349fd81c0edc43036eb41 (127.22.235.253:46613), 10fcd1293d6a4760a23b56da779e63f1 (127.22.235.252:39223)
W20260808 08:29:37.898942 28215 tablet.cc:2369] T 00000000000000000000000000000000 P c1094bde4f7a4dfa8243cded9d788084: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 08:29:37.899606 28421 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a45ebce72e349fd81c0edc43036eb41 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 08:29:37.900022 28592 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 10fcd1293d6a4760a23b56da779e63f1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 08:29:37.900086 28592 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 08:29:37.900372 28382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c1094bde4f7a4dfa8243cded9d788084" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7a45ebce72e349fd81c0edc43036eb41"
I20260808 08:29:37.898437 28433 master_runner.cc:186] Error getting master registration for 127.22.235.254:38009: OK, instance_id { permanent_uuid: "c1094bde4f7a4dfa8243cded9d788084" instance_seqno: 1786177777401837 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 08:29:37.901499 28433 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 08:29:37.903611 28509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c1094bde4f7a4dfa8243cded9d788084" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "10fcd1293d6a4760a23b56da779e63f1"
I20260808 08:29:37.903822 28509 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10fcd1293d6a4760a23b56da779e63f1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:29:37.905175 28509 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 10fcd1293d6a4760a23b56da779e63f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1094bde4f7a4dfa8243cded9d788084 in term 1.
I20260808 08:29:37.906040 28207 leader_election.cc:302] T 00000000000000000000000000000000 P c1094bde4f7a4dfa8243cded9d788084 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10fcd1293d6a4760a23b56da779e63f1, c1094bde4f7a4dfa8243cded9d788084; no voters:
I20260808 08:29:37.906324 28575 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1094bde4f7a4dfa8243cded9d788084 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:29:37.906553 28575 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1094bde4f7a4dfa8243cded9d788084 [term 1 LEADER]: Becoming Leader. State: Replica: c1094bde4f7a4dfa8243cded9d788084, State: Running, Role: LEADER
I20260808 08:29:37.906900 28575 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1094bde4f7a4dfa8243cded9d788084 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1094bde4f7a4dfa8243cded9d788084" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38009 } } peers { permanent_uuid: "7a45ebce72e349fd81c0edc43036eb41" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46613 } } peers { permanent_uuid: "10fcd1293d6a4760a23b56da779e63f1" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 39223 } }
I20260808 08:29:37.908245 28574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10fcd1293d6a4760a23b56da779e63f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1094bde4f7a4dfa8243cded9d788084" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38009 } } peers { permanent_uuid: "7a45ebce72e349fd81c0edc43036eb41" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46613 } } peers { permanent_uuid: "10fcd1293d6a4760a23b56da779e63f1" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 39223 } } }
I20260808 08:29:37.908375 28574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10fcd1293d6a4760a23b56da779e63f1 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 08:29:37.900592 28382 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a45ebce72e349fd81c0edc43036eb41 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:29:37.914093 28563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:37.902774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46304 (local address 127.22.235.252:39223)
0808 08:29:37.902889 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:37.902894 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:37.905371 (+ 2477us) server_negotiation.cc:407] Connection header received
0808 08:29:37.905409 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:37.905411 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:37.905453 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:37.905528 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:37.906449 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:37.907183 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:37.908254 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:37.908632 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:37.908710 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:37.910491 (+ 1781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:37.910495 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:37.910498 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:37.910524 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:37.912200 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:37.912716 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:37.912720 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:37.912722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:37.912811 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:37.913142 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:37.913146 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:37.913147 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:37.913348 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:37.913460 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:37.913809 (+ 349us) server_negotiation.cc:299] Negotiation successful
0808 08:29:37.913927 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 08:29:37.914178 28382 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7a45ebce72e349fd81c0edc43036eb41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1094bde4f7a4dfa8243cded9d788084 in term 1.
I20260808 08:29:37.915307 28595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a45ebce72e349fd81c0edc43036eb41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1094bde4f7a4dfa8243cded9d788084" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38009 } } peers { permanent_uuid: "7a45ebce72e349fd81c0edc43036eb41" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46613 } } peers { permanent_uuid: "10fcd1293d6a4760a23b56da779e63f1" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 39223 } } }
I20260808 08:29:37.915431 28595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a45ebce72e349fd81c0edc43036eb41 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 08:29:37.915467 28575 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1094bde4f7a4dfa8243cded9d788084 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1094bde4f7a4dfa8243cded9d788084. Latest consensus state: current_term: 1 leader_uuid: "c1094bde4f7a4dfa8243cded9d788084" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1094bde4f7a4dfa8243cded9d788084" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38009 } } peers { permanent_uuid: "7a45ebce72e349fd81c0edc43036eb41" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46613 } } peers { permanent_uuid: "10fcd1293d6a4760a23b56da779e63f1" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 39223 } } }
I20260808 08:29:37.915596 28575 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1094bde4f7a4dfa8243cded9d788084 [sys.catalog]: This master's current role is: LEADER
I20260808 08:29:37.908767 28183 master_runner.cc:186] Error getting master registration for 127.22.235.253:46613: OK, instance_id { permanent_uuid: "7a45ebce72e349fd81c0edc43036eb41" instance_seqno: 1786177777623935 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 08:29:37.916119 28183 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 08:29:37.917444 28430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:37.903065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44926 (local address 127.22.235.253:46613)
0808 08:29:37.903167 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:37.903171 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:37.908402 (+ 5231us) server_negotiation.cc:407] Connection header received
0808 08:29:37.908602 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:37.908605 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:37.908679 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:37.908737 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:37.909712 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:37.910441 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:37.911648 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:37.911978 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:37.912052 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:37.913841 (+ 1789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:37.913845 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:37.913848 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:37.913874 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:37.915642 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:37.916122 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:37.916126 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:37.916127 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:37.916214 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:37.916509 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:37.916513 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:37.916515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:37.916752 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:37.916853 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:37.917195 (+ 342us) server_negotiation.cc:299] Negotiation successful
0808 08:29:37.917302 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260808 08:29:37.918243 28653 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7a45ebce72e349fd81c0edc43036eb41: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 08:29:37.918324 28653 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 08:29:37.921546 28306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:37.903262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44696 (local address 127.22.235.254:38009)
0808 08:29:37.903374 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:37.903378 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:37.903623 (+ 245us) server_negotiation.cc:407] Connection header received
0808 08:29:37.903803 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:37.903807 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:37.903853 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:37.903902 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:37.905252 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:37.906089 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:37.907318 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:37.907660 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:37.907738 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:37.917167 (+ 9429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:37.917171 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:37.917173 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:37.917197 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:37.919216 (+ 2019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:37.919604 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:37.919608 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:37.919609 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:37.919693 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:37.920296 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:37.920300 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:37.920302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:37.920512 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:37.920668 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:37.921306 (+ 638us) server_negotiation.cc:299] Negotiation successful
0808 08:29:37.921412 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260808 08:29:37.922668 28650 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c1094bde4f7a4dfa8243cded9d788084: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 08:29:37.922726 28650 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 08:29:37.923053 28657 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:29:37.923914 28657 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:29:37.930980 28509 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 10fcd1293d6a4760a23b56da779e63f1 [term 1 FOLLOWER]: Refusing update from remote peer c1094bde4f7a4dfa8243cded9d788084: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:29:37.931751 28575 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c1094bde4f7a4dfa8243cded9d788084 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10fcd1293d6a4760a23b56da779e63f1" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:29:37.932327 28382 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7a45ebce72e349fd81c0edc43036eb41 [term 1 FOLLOWER]: Refusing update from remote peer c1094bde4f7a4dfa8243cded9d788084: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:29:37.935855 28574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10fcd1293d6a4760a23b56da779e63f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1094bde4f7a4dfa8243cded9d788084. Latest consensus state: current_term: 1 leader_uuid: "c1094bde4f7a4dfa8243cded9d788084" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1094bde4f7a4dfa8243cded9d788084" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38009 } } peers { permanent_uuid: "7a45ebce72e349fd81c0edc43036eb41" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46613 } } peers { permanent_uuid: "10fcd1293d6a4760a23b56da779e63f1" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 39223 } } }
I20260808 08:29:37.935976 28574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10fcd1293d6a4760a23b56da779e63f1 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 08:29:37.936136 28652 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c1094bde4f7a4dfa8243cded9d788084 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a45ebce72e349fd81c0edc43036eb41" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:29:37.939934 28595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a45ebce72e349fd81c0edc43036eb41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1094bde4f7a4dfa8243cded9d788084. Latest consensus state: current_term: 1 leader_uuid: "c1094bde4f7a4dfa8243cded9d788084" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1094bde4f7a4dfa8243cded9d788084" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38009 } } peers { permanent_uuid: "7a45ebce72e349fd81c0edc43036eb41" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46613 } } peers { permanent_uuid: "10fcd1293d6a4760a23b56da779e63f1" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 39223 } } }
I20260808 08:29:37.940073 28595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a45ebce72e349fd81c0edc43036eb41 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 08:29:37.942866 28575 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1094bde4f7a4dfa8243cded9d788084 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c1094bde4f7a4dfa8243cded9d788084" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1094bde4f7a4dfa8243cded9d788084" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38009 } } peers { permanent_uuid: "7a45ebce72e349fd81c0edc43036eb41" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46613 } } peers { permanent_uuid: "10fcd1293d6a4760a23b56da779e63f1" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 39223 } } }
I20260808 08:29:37.942971 28575 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1094bde4f7a4dfa8243cded9d788084 [sys.catalog]: This master's current role is: LEADER
I20260808 08:29:37.943060 28575 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1094bde4f7a4dfa8243cded9d788084 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c1094bde4f7a4dfa8243cded9d788084" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1094bde4f7a4dfa8243cded9d788084" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38009 } } peers { permanent_uuid: "7a45ebce72e349fd81c0edc43036eb41" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46613 } } peers { permanent_uuid: "10fcd1293d6a4760a23b56da779e63f1" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 39223 } } }
I20260808 08:29:37.943120 28575 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1094bde4f7a4dfa8243cded9d788084 [sys.catalog]: This master's current role is: LEADER
I20260808 08:29:37.943183 28595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a45ebce72e349fd81c0edc43036eb41 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c1094bde4f7a4dfa8243cded9d788084" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1094bde4f7a4dfa8243cded9d788084" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38009 } } peers { permanent_uuid: "7a45ebce72e349fd81c0edc43036eb41" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46613 } } peers { permanent_uuid: "10fcd1293d6a4760a23b56da779e63f1" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 39223 } } }
I20260808 08:29:37.943286 28595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a45ebce72e349fd81c0edc43036eb41 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 08:29:37.944175 28574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10fcd1293d6a4760a23b56da779e63f1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c1094bde4f7a4dfa8243cded9d788084" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1094bde4f7a4dfa8243cded9d788084" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 38009 } } peers { permanent_uuid: "7a45ebce72e349fd81c0edc43036eb41" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 46613 } } peers { permanent_uuid: "10fcd1293d6a4760a23b56da779e63f1" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 39223 } } }
I20260808 08:29:37.944276 28574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10fcd1293d6a4760a23b56da779e63f1 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 08:29:37.945952 28665 mvcc.cc:205] Tried to move back new op lower bound from 7316184178396631040 to 7316184178309947392. Current Snapshot: MvccSnapshot[applied={T|T < 7316184178396631040}]
I20260808 08:29:37.950268 28657 catalog_manager.cc:1347] Generated new cluster ID: 5429c3e3ecbd44d298ba516186b9f7e5
I20260808 08:29:37.950338 28657 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:29:37.962275 28657 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:29:37.963495 28657 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:29:37.980577 28657 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1094bde4f7a4dfa8243cded9d788084: Generated new TSK 0
I20260808 08:29:37.981381 28657 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 08:29:38.015911 28306 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:29:38.012804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44710 (local address 127.22.235.254:38009)
0808 08:29:38.012936 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:38.012940 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:38.014156 (+ 1216us) server_negotiation.cc:407] Connection header received
0808 08:29:38.015016 (+ 860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:38.015020 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:38.015078 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:38.015131 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:38.015771 (+ 640us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44710: BlockingRecv error: recv got EOF from 127.0.0.1:44710 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260808 08:29:38.017562 28430 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:29:38.013537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44942 (local address 127.22.235.253:46613)
0808 08:29:38.013648 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:38.013653 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:38.016296 (+ 2643us) server_negotiation.cc:407] Connection header received
0808 08:29:38.016811 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:38.016815 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:38.016867 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:38.016915 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:38.017445 (+ 530us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44942: BlockingRecv error: recv got EOF from 127.0.0.1:44942 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260808 08:29:38.018617 28563 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:29:38.014459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46306 (local address 127.22.235.252:39223)
0808 08:29:38.014592 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:38.014597 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:38.016728 (+ 2131us) server_negotiation.cc:407] Connection header received
0808 08:29:38.017916 (+ 1188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:38.017919 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:38.017981 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:38.018031 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:38.018501 (+ 470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46306: BlockingRecv error: recv got EOF from 127.0.0.1:46306 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260808 08:29:38.023089 28430 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:29:38.021593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44946 (local address 127.22.235.253:46613)
0808 08:29:38.021691 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:38.021695 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:38.022172 (+ 477us) server_negotiation.cc:407] Connection header received
0808 08:29:38.022388 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:38.022391 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:38.022464 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:38.022515 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:38.022994 (+ 479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44946: BlockingRecv error: recv got EOF from 127.0.0.1:44946 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260808 08:29:38.027999 28430 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:29:38.026530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44960 (local address 127.22.235.253:46613)
0808 08:29:38.026627 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:38.026631 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:38.027173 (+ 542us) server_negotiation.cc:407] Connection header received
0808 08:29:38.027343 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:38.027346 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:38.027390 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:38.027436 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:38.027896 (+ 460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44960: BlockingRecv error: recv got EOF from 127.0.0.1:44960 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260808 08:29:38.033280 28430 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:29:38.031819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44966 (local address 127.22.235.253:46613)
0808 08:29:38.031911 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:38.031915 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:38.032426 (+ 511us) server_negotiation.cc:407] Connection header received
0808 08:29:38.032595 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:38.032598 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:38.032676 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:38.032722 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:38.033190 (+ 468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44966: BlockingRecv error: recv got EOF from 127.0.0.1:44966 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260808 08:29:38.040560 28430 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:29:38.039147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44970 (local address 127.22.235.253:46613)
0808 08:29:38.039269 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:38.039273 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:38.039749 (+ 476us) server_negotiation.cc:407] Connection header received
0808 08:29:38.039914 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:38.039917 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:38.039960 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:38.040017 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:38.040471 (+ 454us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44970: BlockingRecv error: recv got EOF from 127.0.0.1:44970 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260808 08:29:38.052486 28430 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:29:38.050885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44986 (local address 127.22.235.253:46613)
0808 08:29:38.051003 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:38.051007 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:38.051554 (+ 547us) server_negotiation.cc:407] Connection header received
0808 08:29:38.051753 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:38.051757 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:38.051805 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:38.051856 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:38.052380 (+ 524us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44986: BlockingRecv error: recv got EOF from 127.0.0.1:44986 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260808 08:29:38.073098 28430 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:29:38.071089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45000 (local address 127.22.235.253:46613)
0808 08:29:38.071221 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:38.071226 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:38.071879 (+ 653us) server_negotiation.cc:407] Connection header received
0808 08:29:38.072140 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:38.072143 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:38.072205 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:38.072261 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:38.072948 (+ 687us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45000: BlockingRecv error: recv got EOF from 127.0.0.1:45000 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260808 08:29:38.109573 28430 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:29:38.107612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45006 (local address 127.22.235.253:46613)
0808 08:29:38.107733 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:38.107737 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:38.108351 (+ 614us) server_negotiation.cc:407] Connection header received
0808 08:29:38.108603 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:38.108607 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:38.108691 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:38.108743 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:38.109427 (+ 684us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45006: BlockingRecv error: recv got EOF from 127.0.0.1:45006 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260808 08:29:38.177162 28430 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:29:38.176041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45018 (local address 127.22.235.253:46613)
0808 08:29:38.176174 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:38.176179 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:38.176407 (+ 228us) server_negotiation.cc:407] Connection header received
0808 08:29:38.176538 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:38.176541 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:38.176592 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:38.176693 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:38.177056 (+ 363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45018: BlockingRecv error: recv got EOF from 127.0.0.1:45018 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260808 08:29:38.308115 28430 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:29:38.307128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45020 (local address 127.22.235.253:46613)
0808 08:29:38.307263 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:38.307267 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:38.307389 (+ 122us) server_negotiation.cc:407] Connection header received
0808 08:29:38.307520 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:38.307523 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:38.307570 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:38.307650 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:38.308013 (+ 363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45020: BlockingRecv error: recv got EOF from 127.0.0.1:45020 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260808 08:29:38.567368 28430 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:29:38.566277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45024 (local address 127.22.235.253:46613)
0808 08:29:38.566428 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:38.566433 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:38.566449 (+ 16us) server_negotiation.cc:407] Connection header received
0808 08:29:38.566616 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:38.566620 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:38.566668 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:38.566737 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:38.567240 (+ 503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45024: BlockingRecv error: recv got EOF from 127.0.0.1:45024 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 08:29:38.901640 28592 catalog_manager.cc:1259] Loaded cluster ID: 5429c3e3ecbd44d298ba516186b9f7e5
I20260808 08:29:38.901726 28592 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 10fcd1293d6a4760a23b56da779e63f1: loading cluster ID for follower catalog manager: success
I20260808 08:29:38.903427 28592 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 10fcd1293d6a4760a23b56da779e63f1: acquiring CA information for follower catalog manager: success
I20260808 08:29:38.904589 28592 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 10fcd1293d6a4760a23b56da779e63f1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 08:29:38.919659 28653 catalog_manager.cc:1259] Loaded cluster ID: 5429c3e3ecbd44d298ba516186b9f7e5
I20260808 08:29:38.919735 28653 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7a45ebce72e349fd81c0edc43036eb41: loading cluster ID for follower catalog manager: success
I20260808 08:29:38.921398 28653 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7a45ebce72e349fd81c0edc43036eb41: acquiring CA information for follower catalog manager: success
I20260808 08:29:38.922509 28653 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7a45ebce72e349fd81c0edc43036eb41: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 08:29:39.084760 28773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:39.081266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45030 (local address 127.22.235.253:46613)
0808 08:29:39.081528 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:39.081532 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:39.081549 (+ 17us) server_negotiation.cc:407] Connection header received
0808 08:29:39.081680 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:39.081682 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:39.081810 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:39.081907 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:29:39.082105 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:39.082623 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:39.083371 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:39.083588 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:39.083649 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:39.083888 (+ 239us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:29:39.084318 (+ 430us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 08:29:39.084425 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:39.084533 (+ 108us) server_negotiation.cc:299] Negotiation successful
0808 08:29:39.084588 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":77,"threads_started":1}
I20260808 08:29:39.090710 28784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:39.087614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46308 (local address 127.22.235.252:39223)
0808 08:29:39.087811 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:39.087815 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:39.087828 (+ 13us) server_negotiation.cc:407] Connection header received
0808 08:29:39.087959 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:39.087961 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:39.088072 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:39.088164 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:29:39.088310 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:39.088852 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:39.089580 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:39.089785 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:39.089848 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:39.090050 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:29:39.090434 (+ 384us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 08:29:39.090524 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:39.090553 (+ 29us) server_negotiation.cc:299] Negotiation successful
0808 08:29:39.090597 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":61,"threads_started":1}
I20260808 08:29:39.097635 28784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:39.093714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46322 (local address 127.22.235.252:39223)
0808 08:29:39.093820 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:39.093824 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:39.094261 (+ 437us) server_negotiation.cc:407] Connection header received
0808 08:29:39.094356 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:39.094359 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:39.094406 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:39.094469 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:29:39.094703 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:39.095436 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:39.096484 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:39.096720 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:39.096807 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:39.097198 (+ 391us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:29:39.097279 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 08:29:39.097346 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:39.097442 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 08:29:39.097500 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 08:29:39.098063 28796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:39.093514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44726 (local address 127.22.235.254:38009)
0808 08:29:39.093793 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:39.093798 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:39.093826 (+ 28us) server_negotiation.cc:407] Connection header received
0808 08:29:39.093904 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:39.093906 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:39.094022 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:39.094118 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:29:39.094349 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:39.095099 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:39.096231 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:39.096599 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:39.096717 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:39.097224 (+ 507us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:29:39.097536 (+ 312us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 08:29:39.097621 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:39.097878 (+ 257us) server_negotiation.cc:299] Negotiation successful
0808 08:29:39.097940 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":70,"threads_started":1}
I20260808 08:29:39.098248 28773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:39.093782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45042 (local address 127.22.235.253:46613)
0808 08:29:39.093881 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:39.093885 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:39.094391 (+ 506us) server_negotiation.cc:407] Connection header received
0808 08:29:39.094529 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:39.094532 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:39.094581 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:39.094625 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:29:39.094958 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:39.095685 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:39.096921 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:39.097233 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:39.097305 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:39.097657 (+ 352us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:29:39.097735 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 08:29:39.097782 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:39.098080 (+ 298us) server_negotiation.cc:299] Negotiation successful
0808 08:29:39.098134 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 08:29:39.101128 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 28183
I20260808 08:29:39.118916 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 28311
I20260808 08:29:39.133917 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 28433
2026-08-08T08:29:39Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3196 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 08:29:39 dist-test-slave-f7th krb5kdc[28804](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 08:29:39 dist-test-slave-f7th krb5kdc[28804](info): set up 2 sockets
Aug 08 08:29:39 dist-test-slave-f7th krb5kdc[28804](info): commencing operation
krb5kdc: starting...
W20260808 08:29:40.308069 23471 mini_kdc.cc:121] Time spent starting KDC: real 1.147s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/krb5kdc/test-user.keytab.
Aug 08 08:29:40 dist-test-slave-f7th krb5kdc[28804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T08:29:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T08:29:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.235.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
Entry for principal kudu/127.22.235.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab.
I20260808 08:29:40.427505 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:35979
--webserver_interface=127.22.235.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:42247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:35979,127.22.235.253:35609,127.22.235.252:36363
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 08:29:40.538149 28825 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 08:29:40.538429 28825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:29:40.538467 28825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:29:40.538534 28825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 08:29:40.542526 28825 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:29:40.542578 28825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:29:40.542604 28825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:29:40.542626 28825 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:29:40.542655 28825 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:29:40.547731 28825 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:42247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:35979,127.22.235.253:35609,127.22.235.252:36363
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.254.keytab
--principal=kudu/127.22.235.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.254:35979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.235.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28825
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:29:40.548758 28825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:29:40.549723 28825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 08:29:40.557870 28825 server_base.cc:1034] running on GCE node
W20260808 08:29:40.557806 28833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:40.557809 28836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:40.557806 28840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:40.558632 28825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:29:40.559685 28825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:29:40.560838 28825 hybrid_clock.cc:648] HybridClock initialized: now 1786177780560819 us; error 30 us; skew 500 ppm
Aug 08 08:29:40 dist-test-slave-f7th krb5kdc[28804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177780, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:29:40.563454 28825 init.cc:377] Logged in from keytab as kudu/127.22.235.254@KRBTEST.COM (short username kudu)
I20260808 08:29:40.564255 28825 webserver.cc:460] Webserver started at http://127.22.235.254:45435/ using document root <none> and password file <none>
I20260808 08:29:40.564800 28825 fs_manager.cc:359] Metadata directory not provided
I20260808 08:29:40.564859 28825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:29:40.565039 28825 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:29:40.566969 28825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-0/data/instance:
uuid: "af48b9c76e7340da879a969e0b91f6f0"
format_stamp: "Formatted at 2026-08-08 08:29:40 on dist-test-slave-f7th"
server_key: "7af0a28050f2d8262da086571daa4ca2"
server_key_iv: "f9b2862afcdec15d51ae77e5d3596a02"
server_key_version: "encryptionkey@0"
I20260808 08:29:40.567466 28825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-0/wal/instance:
uuid: "af48b9c76e7340da879a969e0b91f6f0"
format_stamp: "Formatted at 2026-08-08 08:29:40 on dist-test-slave-f7th"
server_key: "7af0a28050f2d8262da086571daa4ca2"
server_key_iv: "f9b2862afcdec15d51ae77e5d3596a02"
server_key_version: "encryptionkey@0"
I20260808 08:29:40.570966 28825 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 08:29:40.573338 28851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:40.574532 28825 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:29:40.574631 28825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-0/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-0/wal
uuid: "af48b9c76e7340da879a969e0b91f6f0"
format_stamp: "Formatted at 2026-08-08 08:29:40 on dist-test-slave-f7th"
server_key: "7af0a28050f2d8262da086571daa4ca2"
server_key_iv: "f9b2862afcdec15d51ae77e5d3596a02"
server_key_version: "encryptionkey@0"
I20260808 08:29:40.574712 28825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:29:40.589111 28825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:29:40.591673 28825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:29:40.591835 28825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:29:40.598737 28825 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.254:35979
I20260808 08:29:40.599000 28934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.254:35979 every 8 connection(s)
I20260808 08:29:40.599670 28825 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-0/data/info.pb
I20260808 08:29:40.602494 28935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:29:40.603369 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 28825
I20260808 08:29:40.603507 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-0/wal/instance
I20260808 08:29:40.603780 23471 external_mini_cluster.cc:1428] Setting key 50da88aa7ad8f20c078aac7d37806688
I20260808 08:29:40.606410 28935 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.235.254" port: 35979 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.22.235.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.253@KRBTEST.COM" created.
Aug 08 08:29:40 dist-test-slave-f7th krb5kdc[28804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177780, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:29:40.621199 28942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:40.608666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55596 (local address 127.22.235.254:35979)
0808 08:29:40.609087 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:40.609111 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:40.609258 (+ 147us) server_negotiation.cc:407] Connection header received
0808 08:29:40.610228 (+ 970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:40.610233 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:40.610427 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:40.610591 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:40.611424 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:40.612278 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:40.613316 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:40.613686 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:40.613832 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:40.616110 (+ 2278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:40.616116 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:40.616126 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:40.616147 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:40.617966 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:40.618521 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:40.618527 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:40.618528 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:40.618632 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:40.619021 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:40.619024 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:40.619025 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:40.619499 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:40.619636 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:40.620098 (+ 462us) server_negotiation.cc:299] Negotiation successful
0808 08:29:40.620322 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":285,"spinlock_wait_cycles":21760,"thread_start_us":128,"threads_started":1}
I20260808 08:29:40.621201 28940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:40.608665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:35979 (local address 127.0.0.1:55596)
0808 08:29:40.609125 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:40.609162 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:40.609937 (+ 775us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:40.610603 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:40.610609 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:40.611037 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:29:40.611274 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:40.611288 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:40.612402 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:40.612406 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:40.613204 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:40.613216 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:40.613832 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:40.613836 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:40.613999 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:40.614518 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:29:40.614534 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:29:40.615978 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:29:40.618075 (+ 2097us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:29:40.618087 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:29:40.618097 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:29:40.618394 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:29:40.618719 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:29:40.618722 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:29:40.618724 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:29:40.618891 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:29:40.619644 (+ 753us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:29:40.619652 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:29:40.619823 (+ 171us) client_negotiation.cc:769] Sending connection context
0808 08:29:40.620021 (+ 198us) client_negotiation.cc:240] Negotiation successful
0808 08:29:40.620394 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"mutex_wait_us":109,"thread_start_us":112,"threads_started":1}
I20260808 08:29:40.623792 28935 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.235.253" port: 35609 } has no permanent_uuid. Determining permanent_uuid...
I20260808 08:29:40.624472 28940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:40.624243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.253:35609 (local address 127.0.0.1:57768)
0808 08:29:40.624346 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:40.624414 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.253:35609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 08:29:40.624675 28857 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.235.253:35609: connect: Connection refused (error 111)
I20260808 08:29:40.625406 28940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:40.625224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.253:35609 (local address 127.0.0.1:57776)
0808 08:29:40.625317 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:40.625359 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.253:35609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 08:29:40.625538 28935 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.235.253:35609: Network error: Client connection negotiation failed: client connection to 127.22.235.253:35609: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.253.keytab.
Entry for principal kudu/127.22.235.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.253.keytab.
I20260808 08:29:40.630826 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.253:35609
--webserver_interface=127.22.235.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.253.keytab
--principal=kudu/127.22.235.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:42247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:35979,127.22.235.253:35609,127.22.235.252:36363
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 08:29:40.674805 28935 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.235.253" port: 35609 } attempt: 1
I20260808 08:29:40.675709 28940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:40.675402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.253:35609 (local address 127.0.0.1:45706)
0808 08:29:40.675565 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:40.675625 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.253:35609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 08:29:40.676383 28940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:40.676168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.253:35609 (local address 127.0.0.1:45708)
0808 08:29:40.676274 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:40.676318 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.253:35609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 08:29:40.676530 28935 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.235.253:35609: Network error: Client connection negotiation failed: client connection to 127.22.235.253:35609: connect: Connection refused (error 111)
W20260808 08:29:40.742103 28948 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 08:29:40.742383 28948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:29:40.742420 28948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:29:40.742475 28948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 08:29:40.744689 28935 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.235.253" port: 35609 } attempt: 2
I20260808 08:29:40.745579 28940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:40.745188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.253:35609 (local address 127.0.0.1:45720)
0808 08:29:40.745372 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:40.745460 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.253:35609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260808 08:29:40.746426 28940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:40.746146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.253:35609 (local address 127.0.0.1:45728)
0808 08:29:40.746291 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:40.746355 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.253:35609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 08:29:40.746564 28935 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.235.253:35609: Network error: Client connection negotiation failed: client connection to 127.22.235.253:35609: connect: Connection refused (error 111)
W20260808 08:29:40.746815 28948 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:29:40.746857 28948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:29:40.746874 28948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:29:40.746896 28948 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:29:40.746914 28948 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:29:40.752043 28948 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:42247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:35979,127.22.235.253:35609,127.22.235.252:36363
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.253.keytab
--principal=kudu/127.22.235.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.253:35609
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.235.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28948
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:29:40.753037 28948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:29:40.753996 28948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:29:40.762212 28965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:40.762234 28956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:40.762252 28948 server_base.cc:1034] running on GCE node
W20260808 08:29:40.762440 28958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:40.762923 28948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:29:40.763979 28948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:29:40.765141 28948 hybrid_clock.cc:648] HybridClock initialized: now 1786177780765119 us; error 27 us; skew 500 ppm
Aug 08 08:29:40 dist-test-slave-f7th krb5kdc[28804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177780, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:29:40.767755 28948 init.cc:377] Logged in from keytab as kudu/127.22.235.253@KRBTEST.COM (short username kudu)
I20260808 08:29:40.768664 28948 webserver.cc:460] Webserver started at http://127.22.235.253:39811/ using document root <none> and password file <none>
I20260808 08:29:40.769197 28948 fs_manager.cc:359] Metadata directory not provided
I20260808 08:29:40.769264 28948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:29:40.769485 28948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:29:40.771452 28948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-1/data/instance:
uuid: "7195aacdff92472d9b183fcdf0baaead"
format_stamp: "Formatted at 2026-08-08 08:29:40 on dist-test-slave-f7th"
server_key: "51f01813b5cad3afddbb9a53d4f7346a"
server_key_iv: "09df1f33bbd68ac5e38a5426ca5a0743"
server_key_version: "encryptionkey@0"
I20260808 08:29:40.772033 28948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-1/wal/instance:
uuid: "7195aacdff92472d9b183fcdf0baaead"
format_stamp: "Formatted at 2026-08-08 08:29:40 on dist-test-slave-f7th"
server_key: "51f01813b5cad3afddbb9a53d4f7346a"
server_key_iv: "09df1f33bbd68ac5e38a5426ca5a0743"
server_key_version: "encryptionkey@0"
I20260808 08:29:40.775578 28948 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 08:29:40.777976 28975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:40.779224 28948 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 08:29:40.779342 28948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-1/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-1/wal
uuid: "7195aacdff92472d9b183fcdf0baaead"
format_stamp: "Formatted at 2026-08-08 08:29:40 on dist-test-slave-f7th"
server_key: "51f01813b5cad3afddbb9a53d4f7346a"
server_key_iv: "09df1f33bbd68ac5e38a5426ca5a0743"
server_key_version: "encryptionkey@0"
I20260808 08:29:40.779439 28948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:29:40.807384 28948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:29:40.810521 28948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:29:40.810702 28948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:29:40.817332 28948 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.253:35609
I20260808 08:29:40.817607 29063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.253:35609 every 8 connection(s)
I20260808 08:29:40.818262 28948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-1/data/info.pb
I20260808 08:29:40.821007 29064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:29:40.824672 29064 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.235.254" port: 35979 } has no permanent_uuid. Determining permanent_uuid...
I20260808 08:29:40.827035 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 28948
I20260808 08:29:40.827167 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-1/wal/instance
I20260808 08:29:40.827486 23471 external_mini_cluster.cc:1428] Setting key 7bda32399fe0f985f791b079fedd1e40
Aug 08 08:29:40 dist-test-slave-f7th krb5kdc[28804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177780, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.253@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.235.252@KRBTEST.COM; defaulting to no policy
I20260808 08:29:40.837699 28935 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.235.253" port: 35609 } attempt: 3
I20260808 08:29:40.839358 28942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:40.826449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45426 (local address 127.22.235.254:35979)
0808 08:29:40.826584 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:40.826588 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:40.827463 (+ 875us) server_negotiation.cc:407] Connection header received
0808 08:29:40.828592 (+ 1129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:40.828595 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:40.828675 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:40.828748 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:40.830126 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:40.830882 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:40.832171 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:40.832527 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:40.832669 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:40.835378 (+ 2709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:40.835382 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:40.835385 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:40.835416 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:40.836955 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:40.837418 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:40.837420 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:40.837421 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:40.837497 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:40.837831 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:40.837835 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:40.837836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:40.838096 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:40.838199 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:40.839042 (+ 843us) server_negotiation.cc:299] Negotiation successful
0808 08:29:40.839200 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.235.252@KRBTEST.COM" created.
I20260808 08:29:40.840371 29072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:40.826777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:35979 (local address 127.0.0.1:45426)
0808 08:29:40.827286 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:40.827344 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:40.828352 (+ 1008us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:40.828939 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:40.828947 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:40.829400 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:29:40.829980 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:40.829994 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:40.831086 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:40.831089 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:40.832035 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:40.832047 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:40.832743 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:40.832746 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:40.832943 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:40.833538 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:29:40.833554 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:29:40.835200 (+ 1646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:29:40.837072 (+ 1872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:29:40.837078 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:29:40.837090 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:29:40.837336 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:29:40.837564 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:29:40.837566 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:29:40.837568 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:29:40.837740 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:29:40.838442 (+ 702us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:29:40.838449 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:29:40.838774 (+ 325us) client_negotiation.cc:769] Sending connection context
0808 08:29:40.838978 (+ 204us) client_negotiation.cc:240] Negotiation successful
0808 08:29:40.839371 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":52,"thread_start_us":113,"threads_started":1}
I20260808 08:29:40.842342 29064 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.235.253" port: 35609 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 08:29:40 dist-test-slave-f7th krb5kdc[28804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177780, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for kudu/127.22.235.253@KRBTEST.COM
Aug 08 08:29:40 dist-test-slave-f7th krb5kdc[28804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177780, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.253@KRBTEST.COM for kudu/127.22.235.253@KRBTEST.COM
I20260808 08:29:40.849146 28940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:40.838122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.253:35609 (local address 127.0.0.1:45744)
0808 08:29:40.838252 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:40.838266 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:40.838337 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:40.839082 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:40.839085 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:40.839274 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:29:40.839366 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:40.839373 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:40.840729 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:40.840732 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:40.841498 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:40.841509 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:40.842247 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:40.842251 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:40.842353 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:40.842772 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:29:40.842784 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:29:40.844149 (+ 1365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:29:40.847351 (+ 3202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:29:40.847354 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:29:40.847356 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:29:40.847600 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:29:40.847945 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:29:40.847948 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:29:40.847950 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:29:40.848052 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:29:40.848600 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:29:40.848603 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:29:40.848749 (+ 146us) client_negotiation.cc:769] Sending connection context
0808 08:29:40.848846 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 08:29:40.848985 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 08:29:40.849288 29075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:40.838305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45744 (local address 127.22.235.253:35609)
0808 08:29:40.838585 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:40.838597 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:40.838620 (+ 23us) server_negotiation.cc:407] Connection header received
0808 08:29:40.838658 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:40.838664 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:40.838905 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:40.839089 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:40.839498 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:40.840380 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:40.841636 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:40.842108 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:40.842244 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:40.844319 (+ 2075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:40.844332 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:40.844347 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:40.844373 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:40.847222 (+ 2849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:40.847741 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:40.847747 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:40.847748 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:40.847836 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:40.848170 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:40.848173 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:40.848175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:40.848474 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:40.848641 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:40.848956 (+ 315us) server_negotiation.cc:299] Negotiation successful
0808 08:29:40.849110 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":183,"thread_start_us":75,"threads_started":1}
I20260808 08:29:40.851147 28935 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.235.252" port: 36363 } has no permanent_uuid. Determining permanent_uuid...
I20260808 08:29:40.851899 28940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:40.851606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:36363 (local address 127.0.0.1:40100)
0808 08:29:40.851789 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:40.851838 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.252:36363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 08:29:40.851938 29072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:40.843130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.253:35609 (local address 127.0.0.1:45760)
0808 08:29:40.843394 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:40.843408 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:40.843485 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:40.843819 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:40.843822 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:40.843986 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:29:40.844071 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:40.844077 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:40.844878 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:40.844881 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:40.845664 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:40.845676 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:40.846102 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:40.846106 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:40.846370 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:40.846720 (+ 350us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:29:40.846731 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:29:40.847887 (+ 1156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:29:40.850560 (+ 2673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:29:40.850564 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:29:40.850565 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:29:40.850793 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:29:40.851113 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:29:40.851115 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:29:40.851117 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:29:40.851202 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:29:40.851531 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:29:40.851534 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:29:40.851628 (+ 94us) client_negotiation.cc:769] Sending connection context
0808 08:29:40.851731 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 08:29:40.851835 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197}
I20260808 08:29:40.852362 29077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:40.843163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45760 (local address 127.22.235.253:35609)
0808 08:29:40.843652 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:40.843657 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:40.843665 (+ 8us) server_negotiation.cc:407] Connection header received
0808 08:29:40.843687 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:40.843688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:40.843716 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:40.843786 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:40.844174 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:40.844785 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:40.845776 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:40.846000 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:40.846089 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:40.848035 (+ 1946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:40.848038 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:40.848040 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:40.848060 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:40.850439 (+ 2379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:40.850883 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:40.850885 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:40.850886 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:40.851013 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:40.851296 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:40.851298 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:40.851299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:40.851443 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:40.851515 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:40.852119 (+ 604us) server_negotiation.cc:299] Negotiation successful
0808 08:29:40.852220 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":170,"threads_started":1}
I20260808 08:29:40.852854 28940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:40.852366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:36363 (local address 127.0.0.1:40108)
0808 08:29:40.852740 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:40.852790 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.252:36363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313}
I20260808 08:29:40.852986 29064 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.235.252" port: 36363 } has no permanent_uuid. Determining permanent_uuid...
W20260808 08:29:40.853009 28935 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.235.252:36363: Network error: Client connection negotiation failed: client connection to 127.22.235.252:36363: connect: Connection refused (error 111)
I20260808 08:29:40.853600 29072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:40.853378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:36363 (local address 127.0.0.1:40118)
0808 08:29:40.853480 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:40.853546 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.252:36363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 08:29:40.853789 28980 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.235.252:36363: connect: Connection refused (error 111)
I20260808 08:29:40.854576 29072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:40.854401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:36363 (local address 127.0.0.1:40132)
0808 08:29:40.854492 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:40.854532 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.252:36363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 08:29:40.854780 29064 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.235.252:36363: Network error: Client connection negotiation failed: client connection to 127.22.235.252:36363: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.235.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.252.keytab.
Entry for principal kudu/127.22.235.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.252.keytab.
I20260808 08:29:40.857151 23471 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
/tmp/dist-test-taskGDnrCb/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.252:36363
--webserver_interface=127.22.235.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.252.keytab
--principal=kudu/127.22.235.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.235.212:42247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.235.254:35979,127.22.235.253:35609,127.22.235.252:36363
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 08:29:40.884187 28935 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.235.252" port: 36363 } attempt: 1
I20260808 08:29:40.885032 28940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:40.884738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:36363 (local address 127.0.0.1:40140)
0808 08:29:40.884880 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:40.884941 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.252:36363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 08:29:40.885751 28940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:40.885537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:36363 (local address 127.0.0.1:40148)
0808 08:29:40.885642 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:40.885689 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.252:36363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 08:29:40.885897 28935 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.235.252:36363: Network error: Client connection negotiation failed: client connection to 127.22.235.252:36363: connect: Connection refused (error 111)
I20260808 08:29:40.904044 29064 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.235.252" port: 36363 } attempt: 1
I20260808 08:29:40.904901 29072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:40.904586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:36363 (local address 127.0.0.1:40164)
0808 08:29:40.904738 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:40.904811 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.252:36363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260808 08:29:40.905759 29072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:40.905537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:36363 (local address 127.0.0.1:40180)
0808 08:29:40.905642 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:40.905692 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.252:36363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 08:29:40.905885 29064 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.235.252:36363: Network error: Client connection negotiation failed: client connection to 127.22.235.252:36363: connect: Connection refused (error 111)
I20260808 08:29:40.961067 28935 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.235.252" port: 36363 } attempt: 2
I20260808 08:29:40.961926 28940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:40.961617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:36363 (local address 127.0.0.1:40190)
0808 08:29:40.961769 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:40.961831 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.252:36363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 08:29:40.962694 28940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:40.962474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:36363 (local address 127.0.0.1:40194)
0808 08:29:40.962584 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:40.962631 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.252:36363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 08:29:40.962819 28935 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.235.252:36363: Network error: Client connection negotiation failed: client connection to 127.22.235.252:36363: connect: Connection refused (error 111)
W20260808 08:29:40.971374 29080 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 08:29:40.971647 29080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 08:29:40.971691 29080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 08:29:40.971753 29080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 08:29:40.974035 29064 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.235.252" port: 36363 } attempt: 2
I20260808 08:29:40.974771 29072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:40.974509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:36363 (local address 127.0.0.1:40200)
0808 08:29:40.974639 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:40.974697 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.252:36363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 08:29:40.975462 29072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:40.975227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:36363 (local address 127.0.0.1:40216)
0808 08:29:40.975330 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:40.975405 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.235.252:36363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 08:29:40.975574 29064 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.235.252:36363: Network error: Client connection negotiation failed: client connection to 127.22.235.252:36363: connect: Connection refused (error 111)
W20260808 08:29:40.976006 29080 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 08:29:40.976058 29080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 08:29:40.976092 29080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 08:29:40.976122 29080 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 08:29:40.976150 29080 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 08:29:40.981526 29080 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.235.212:42247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.22.235.254:35979,127.22.235.253:35609,127.22.235.252:36363
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/krb5kdc/kudu_127.22.235.252.keytab
--principal=kudu/127.22.235.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.235.252:36363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.235.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29080
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 07:34:22 UTC on ccycloud.quasar-bqoudn.root.comops.site
build id 4574264
I20260808 08:29:40.982626 29080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 08:29:40.983685 29080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 08:29:40.991869 29090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:40.991919 29096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 08:29:40.991881 29091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 08:29:40.992079 29080 server_base.cc:1034] running on GCE node
I20260808 08:29:40.992740 29080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 08:29:40.993824 29080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 08:29:40.995000 29080 hybrid_clock.cc:648] HybridClock initialized: now 1786177780994983 us; error 30 us; skew 500 ppm
Aug 08 08:29:40 dist-test-slave-f7th krb5kdc[28804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177780, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 08:29:40.997715 29080 init.cc:377] Logged in from keytab as kudu/127.22.235.252@KRBTEST.COM (short username kudu)
I20260808 08:29:40.998567 29080 webserver.cc:460] Webserver started at http://127.22.235.252:38737/ using document root <none> and password file <none>
I20260808 08:29:40.999104 29080 fs_manager.cc:359] Metadata directory not provided
I20260808 08:29:40.999171 29080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 08:29:40.999377 29080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 08:29:41.001561 29080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-2/data/instance:
uuid: "0b5e0121090f4ea88ebee32253a84632"
format_stamp: "Formatted at 2026-08-08 08:29:40 on dist-test-slave-f7th"
server_key: "31ff5549dd20ca6e76862be4f35ad10f"
server_key_iv: "a6ee79bb1f6bba73111b5dca64f66495"
server_key_version: "encryptionkey@0"
I20260808 08:29:41.002238 29080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-2/wal/instance:
uuid: "0b5e0121090f4ea88ebee32253a84632"
format_stamp: "Formatted at 2026-08-08 08:29:40 on dist-test-slave-f7th"
server_key: "31ff5549dd20ca6e76862be4f35ad10f"
server_key_iv: "a6ee79bb1f6bba73111b5dca64f66495"
server_key_version: "encryptionkey@0"
I20260808 08:29:41.005952 29080 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 08:29:41.008365 29106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 08:29:41.009608 29080 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 08:29:41.009725 29080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-2/data,/tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-2/wal
uuid: "0b5e0121090f4ea88ebee32253a84632"
format_stamp: "Formatted at 2026-08-08 08:29:40 on dist-test-slave-f7th"
server_key: "31ff5549dd20ca6e76862be4f35ad10f"
server_key_iv: "a6ee79bb1f6bba73111b5dca64f66495"
server_key_version: "encryptionkey@0"
I20260808 08:29:41.009821 29080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 08:29:41.034968 29080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 08:29:41.037882 29080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 08:29:41.038074 29080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 08:29:41.045044 29080 rpc_server.cc:307] RPC server started. Bound to: 127.22.235.252:36363
I20260808 08:29:41.045332 29193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.235.252:36363 every 8 connection(s)
I20260808 08:29:41.046096 29080 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-2/data/info.pb
I20260808 08:29:41.049032 29197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 08:29:41.052758 29197 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.235.254" port: 35979 } has no permanent_uuid. Determining permanent_uuid...
I20260808 08:29:41.053725 23471 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu as pid 29080
I20260808 08:29:41.053853 23471 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGDnrCb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786177493204855-23471-0/minicluster-data/master-2/wal/instance
I20260808 08:29:41.054119 23471 external_mini_cluster.cc:1428] Setting key 1bd57f63f70ae0445cac01ced970fb25
I20260808 08:29:41.061993 28935 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.235.252" port: 36363 } attempt: 3
Aug 08 08:29:41 dist-test-slave-f7th krb5kdc[28804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177780, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.252@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
Aug 08 08:29:41 dist-test-slave-f7th krb5kdc[28804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.235.252@KRBTEST.COM
I20260808 08:29:41.066743 28942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:41.054628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45432 (local address 127.22.235.254:35979)
0808 08:29:41.054764 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:41.054769 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:41.055667 (+ 898us) server_negotiation.cc:407] Connection header received
0808 08:29:41.056342 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:41.056346 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:41.056393 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:41.056489 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:41.057577 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:41.058347 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:41.059618 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:41.059944 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:41.060063 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:41.062903 (+ 2840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:41.062907 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:41.062910 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:41.062939 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:41.064592 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:41.065056 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:41.065060 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:41.065061 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:41.065154 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:41.065550 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:41.065554 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:41.065555 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:41.065737 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:41.065826 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:41.066442 (+ 616us) server_negotiation.cc:299] Negotiation successful
0808 08:29:41.066580 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 08:29:41.066712 29064 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.235.252" port: 36363 } attempt: 3
Aug 08 08:29:41 dist-test-slave-f7th krb5kdc[28804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177780, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.254@KRBTEST.COM for kudu/127.22.235.252@KRBTEST.COM
I20260808 08:29:41.067926 29205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:41.055046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.254:35979 (local address 127.0.0.1:45432)
0808 08:29:41.055531 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:41.055575 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:41.056147 (+ 572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:41.056547 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:41.056552 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:41.057091 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:29:41.057442 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:41.057463 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:41.058446 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:41.058448 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:41.059508 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:41.059519 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:41.060123 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:41.060126 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:41.060302 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:41.060931 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:29:41.060953 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:29:41.062735 (+ 1782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:29:41.064702 (+ 1967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:29:41.064711 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:29:41.064721 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:29:41.064960 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:29:41.065245 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:29:41.065248 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:29:41.065250 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:29:41.065439 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:29:41.065915 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:29:41.065922 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:29:41.066182 (+ 260us) client_negotiation.cc:769] Sending connection context
0808 08:29:41.066352 (+ 170us) client_negotiation.cc:240] Negotiation successful
0808 08:29:41.066661 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"mutex_wait_us":5,"spinlock_wait_cycles":13568,"thread_start_us":156,"threads_started":1}
I20260808 08:29:41.069101 29203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:41.055045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40222 (local address 127.22.235.252:36363)
0808 08:29:41.055432 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:41.055447 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:41.055489 (+ 42us) server_negotiation.cc:407] Connection header received
0808 08:29:41.056135 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:41.056153 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:41.056377 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:41.056553 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:41.057412 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:41.058593 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:41.059674 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:41.060272 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:41.060374 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:41.063635 (+ 3261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:41.063643 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:41.063656 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:41.063681 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:41.065869 (+ 2188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:41.066253 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:41.066259 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:41.066261 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:41.066345 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:41.066963 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:41.066967 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:41.066969 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:41.067273 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:41.067489 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:41.068300 (+ 811us) server_negotiation.cc:299] Negotiation successful
0808 08:29:41.068440 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":258,"spinlock_wait_cycles":6400,"thread_start_us":113,"threads_started":1}
I20260808 08:29:41.069442 29197 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.235.253" port: 35609 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 08:29:41 dist-test-slave-f7th krb5kdc[28804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177780, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.253@KRBTEST.COM for kudu/127.22.235.252@KRBTEST.COM
I20260808 08:29:41.073415 28940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:41.062394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:36363 (local address 127.0.0.1:40232)
0808 08:29:41.062887 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:41.062904 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:41.062986 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:41.063216 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:41.063219 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:41.063377 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:29:41.063451 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:41.063456 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:41.064319 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:41.064323 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:41.064979 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:41.064989 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:41.065489 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:41.065493 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:41.065584 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:41.066283 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:29:41.066296 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:29:41.068154 (+ 1858us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:29:41.070932 (+ 2778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:29:41.070936 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:29:41.070938 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:29:41.071190 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:29:41.072232 (+ 1042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:29:41.072235 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:29:41.072237 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:29:41.072328 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:29:41.072952 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:29:41.072956 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:29:41.073061 (+ 105us) client_negotiation.cc:769] Sending connection context
0808 08:29:41.073139 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 08:29:41.073263 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 08:29:41.074025 29206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:41.062666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40232 (local address 127.22.235.252:36363)
0808 08:29:41.062971 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:41.062979 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:41.062992 (+ 13us) server_negotiation.cc:407] Connection header received
0808 08:29:41.063070 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:41.063072 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:41.063112 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:41.063184 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:41.063555 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:41.064215 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:41.065089 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:41.065370 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:41.065472 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:41.068283 (+ 2811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:41.068287 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:41.068289 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:41.068657 (+ 368us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:41.070825 (+ 2168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:41.072011 (+ 1186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:41.072014 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:41.072016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:41.072130 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:41.072446 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:41.072449 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:41.072451 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:41.072679 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:41.073670 (+ 991us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:41.073742 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 08:29:41.073848 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":104,"threads_started":1}
Aug 08 08:29:41 dist-test-slave-f7th krb5kdc[28804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177780, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.252@KRBTEST.COM for kudu/127.22.235.253@KRBTEST.COM
I20260808 08:29:41.077553 29072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:41.067701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:36363 (local address 127.0.0.1:40248)
0808 08:29:41.067807 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:41.067820 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:41.067902 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:41.068454 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:41.068458 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:41.068672 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:29:41.068782 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:41.068790 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:41.069816 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:41.069820 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:41.070590 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:41.070604 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:41.071176 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:41.071180 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:41.071267 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:41.072249 (+ 982us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:29:41.072264 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:29:41.073881 (+ 1617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:29:41.075987 (+ 2106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:29:41.075990 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:29:41.075992 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:29:41.076224 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:29:41.076536 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:29:41.076539 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:29:41.076541 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:29:41.076653 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:29:41.077086 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:29:41.077090 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:29:41.077186 (+ 96us) client_negotiation.cc:769] Sending connection context
0808 08:29:41.077264 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 08:29:41.077390 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 08:29:41.077677 28935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af48b9c76e7340da879a969e0b91f6f0: Bootstrap starting.
Aug 08 08:29:41 dist-test-slave-f7th krb5kdc[28804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.235.254@KRBTEST.COM
I20260808 08:29:41.078080 29208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:41.067734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40248 (local address 127.22.235.252:36363)
0808 08:29:41.068217 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:41.068225 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:41.068237 (+ 12us) server_negotiation.cc:407] Connection header received
0808 08:29:41.068276 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:41.068278 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:41.068316 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:41.068898 (+ 582us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:41.068995 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:41.069690 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:41.070745 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:41.071061 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:41.071162 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:41.074025 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:41.074029 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:41.074032 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:41.074056 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:41.075861 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:41.076345 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:41.076348 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:41.076350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:41.076431 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:41.076759 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:41.076762 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:41.076764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:41.076966 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:41.077708 (+ 742us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:41.077804 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 08:29:41.077910 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":106,"threads_started":1}
I20260808 08:29:41.080125 29205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:41.069879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.253:35609 (local address 127.0.0.1:45774)
0808 08:29:41.070727 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:41.070742 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:41.070869 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:41.071191 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:41.071194 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:41.071354 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:29:41.071453 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:41.071460 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:41.072207 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:41.072211 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:41.073009 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:41.073022 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:41.073723 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:41.073726 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:41.073839 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:41.074528 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:29:41.074541 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:29:41.076202 (+ 1661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:29:41.078557 (+ 2355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:29:41.078560 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:29:41.078562 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:29:41.078921 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:29:41.079171 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:29:41.079174 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:29:41.079176 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:29:41.079273 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:29:41.079687 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:29:41.079690 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:29:41.079773 (+ 83us) client_negotiation.cc:769] Sending connection context
0808 08:29:41.079865 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 08:29:41.079983 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":1}
I20260808 08:29:41.080166 29077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:41.069962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45774 (local address 127.22.235.253:35609)
0808 08:29:41.070061 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:41.070065 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:41.070804 (+ 739us) server_negotiation.cc:407] Connection header received
0808 08:29:41.070975 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:41.070978 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:41.071034 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:41.071075 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:41.071566 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:41.072104 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:41.073155 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:41.073506 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:41.073579 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:41.076355 (+ 2776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:41.076358 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:41.076361 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:41.076382 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:41.077929 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:41.079007 (+ 1078us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:41.079010 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:41.079012 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:41.079090 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:41.079386 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:41.079389 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:41.079391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:41.079592 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:41.079678 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:41.079929 (+ 251us) server_negotiation.cc:299] Negotiation successful
0808 08:29:41.080036 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 08:29:41.081041 29197 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.235.252" port: 36363 } has no permanent_uuid. Determining permanent_uuid...
I20260808 08:29:41.081269 29064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7195aacdff92472d9b183fcdf0baaead: Bootstrap starting.
I20260808 08:29:41.082041 28942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:41.071676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45440 (local address 127.22.235.254:35979)
0808 08:29:41.071786 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:41.071790 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:41.071838 (+ 48us) server_negotiation.cc:407] Connection header received
0808 08:29:41.071965 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:41.071968 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:41.072011 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:41.072075 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:41.072878 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:41.073530 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:41.074748 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:41.075213 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:41.075316 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:41.078493 (+ 3177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:41.078496 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:41.078499 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:41.078523 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:41.080048 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:41.080523 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:41.080526 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:41.080528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:41.080746 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:41.081069 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:41.081072 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:41.081073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:41.081277 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:41.081453 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:41.081654 (+ 201us) server_negotiation.cc:299] Negotiation successful
0808 08:29:41.081763 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 08:29:41.083797 29064 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7195aacdff92472d9b183fcdf0baaead: Neither blocks nor log segments found. Creating new log.
I20260808 08:29:41.084527 29064 log.cc:824] T 00000000000000000000000000000000 P 7195aacdff92472d9b183fcdf0baaead: Log is configured to *not* fsync() on all Append() calls
I20260808 08:29:41.086571 28935 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af48b9c76e7340da879a969e0b91f6f0: Neither blocks nor log segments found. Creating new log.
I20260808 08:29:41.087328 28935 log.cc:824] T 00000000000000000000000000000000 P af48b9c76e7340da879a969e0b91f6f0: Log is configured to *not* fsync() on all Append() calls
Aug 08 08:29:41 dist-test-slave-f7th krb5kdc[28804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.235.253@KRBTEST.COM
I20260808 08:29:41.089073 29064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7195aacdff92472d9b183fcdf0baaead: No bootstrap required, opened a new log
Aug 08 08:29:41 dist-test-slave-f7th krb5kdc[28804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786177780, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.235.252@KRBTEST.COM for kudu/127.22.235.252@KRBTEST.COM
I20260808 08:29:41.092087 29077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:41.083251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45790 (local address 127.22.235.253:35609)
0808 08:29:41.083352 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:41.083357 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:41.083398 (+ 41us) server_negotiation.cc:407] Connection header received
0808 08:29:41.083526 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:41.083529 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:41.083567 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:41.083628 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:41.084258 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:41.084787 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:41.085735 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:41.085963 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:41.086026 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:41.088757 (+ 2731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:41.088761 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:41.088764 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:41.088799 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:41.090350 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:41.090813 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:41.090817 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:41.090819 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:41.090909 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:41.091215 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:41.091219 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:41.091220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:41.091430 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:41.091604 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:41.091769 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 08:29:41.091957 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 08:29:41.092952 29064 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7195aacdff92472d9b183fcdf0baaead [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af48b9c76e7340da879a969e0b91f6f0" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 35979 } } peers { permanent_uuid: "7195aacdff92472d9b183fcdf0baaead" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 35609 } } peers { permanent_uuid: "0b5e0121090f4ea88ebee32253a84632" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 36363 } }
I20260808 08:29:41.093189 29064 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7195aacdff92472d9b183fcdf0baaead [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7195aacdff92472d9b183fcdf0baaead, State: Initialized, Role: FOLLOWER
I20260808 08:29:41.093672 29064 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7195aacdff92472d9b183fcdf0baaead [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af48b9c76e7340da879a969e0b91f6f0" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 35979 } } peers { permanent_uuid: "7195aacdff92472d9b183fcdf0baaead" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 35609 } } peers { permanent_uuid: "0b5e0121090f4ea88ebee32253a84632" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 36363 } }
I20260808 08:29:41.094497 29205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:41.081481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.235.252:36363 (local address 127.0.0.1:40262)
0808 08:29:41.081906 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0808 08:29:41.081917 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 08:29:41.081974 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 08:29:41.085416 (+ 3442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 08:29:41.085418 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 08:29:41.085571 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0808 08:29:41.085638 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:41.085644 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:41.086877 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:41.086881 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:41.087700 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 08:29:41.087712 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 08:29:41.088124 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 08:29:41.088127 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 08:29:41.088236 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:41.088663 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 08:29:41.088677 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 08:29:41.090049 (+ 1372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 08:29:41.092981 (+ 2932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:29:41.092984 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:29:41.092986 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:29:41.093362 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:29:41.093623 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 08:29:41.093626 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 08:29:41.093628 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 08:29:41.093697 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 08:29:41.094085 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 08:29:41.094088 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 08:29:41.094166 (+ 78us) client_negotiation.cc:769] Sending connection context
0808 08:29:41.094239 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 08:29:41.094352 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"mutex_wait_us":20}
I20260808 08:29:41.095080 29208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:41.085071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40262 (local address 127.22.235.252:36363)
0808 08:29:41.085175 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:41.085178 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:41.085190 (+ 12us) server_negotiation.cc:407] Connection header received
0808 08:29:41.085221 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:41.085224 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:41.085260 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:41.085338 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:41.085755 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:41.086744 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:41.087816 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:41.088034 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:41.088117 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:41.090192 (+ 2075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:41.090196 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:41.090198 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:41.090222 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:41.092867 (+ 2645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:41.093464 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:41.093467 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:41.093469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:41.093543 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:41.093785 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:41.093787 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:41.093789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:41.093988 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:41.094759 (+ 771us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:41.094829 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 08:29:41.094935 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":46}
I20260808 08:29:41.095116 29064 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7195aacdff92472d9b183fcdf0baaead [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:29:41.095160 28935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af48b9c76e7340da879a969e0b91f6f0: No bootstrap required, opened a new log
I20260808 08:29:41.095852 23471 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 08:29:41.098039 29197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b5e0121090f4ea88ebee32253a84632: Bootstrap starting.
I20260808 08:29:41.097168 28897 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "af48b9c76e7340da879a969e0b91f6f0"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45440
I20260808 08:29:41.099242 29211 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7195aacdff92472d9b183fcdf0baaead [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af48b9c76e7340da879a969e0b91f6f0" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 35979 } } peers { permanent_uuid: "7195aacdff92472d9b183fcdf0baaead" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 35609 } } peers { permanent_uuid: "0b5e0121090f4ea88ebee32253a84632" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 36363 } } }
I20260808 08:29:41.099377 29211 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7195aacdff92472d9b183fcdf0baaead [sys.catalog]: This master's current role is: FOLLOWER
I20260808 08:29:41.100666 29197 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b5e0121090f4ea88ebee32253a84632: Neither blocks nor log segments found. Creating new log.
W20260808 08:29:41.101043 29225 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7195aacdff92472d9b183fcdf0baaead: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 08:29:41.101158 29225 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 08:29:41.101480 29197 log.cc:824] T 00000000000000000000000000000000 P 0b5e0121090f4ea88ebee32253a84632: Log is configured to *not* fsync() on all Append() calls
I20260808 08:29:41.099030 28948 master_runner.cc:186] Error getting master registration for 127.22.235.254:35979: OK, instance_id { permanent_uuid: "af48b9c76e7340da879a969e0b91f6f0" instance_seqno: 1786177780596867 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 08:29:41.101701 28948 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 08:29:41.103549 29197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b5e0121090f4ea88ebee32253a84632: No bootstrap required, opened a new log
I20260808 08:29:41.104153 28935 raft_consensus.cc:348] T 00000000000000000000000000000000 P af48b9c76e7340da879a969e0b91f6f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af48b9c76e7340da879a969e0b91f6f0" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 35979 } } peers { permanent_uuid: "7195aacdff92472d9b183fcdf0baaead" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 35609 } } peers { permanent_uuid: "0b5e0121090f4ea88ebee32253a84632" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 36363 } }
I20260808 08:29:41.104386 28935 raft_consensus.cc:729] T 00000000000000000000000000000000 P af48b9c76e7340da879a969e0b91f6f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af48b9c76e7340da879a969e0b91f6f0, State: Initialized, Role: FOLLOWER
I20260808 08:29:41.104799 28935 consensus_queue.cc:260] T 00000000000000000000000000000000 P af48b9c76e7340da879a969e0b91f6f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af48b9c76e7340da879a969e0b91f6f0" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 35979 } } peers { permanent_uuid: "7195aacdff92472d9b183fcdf0baaead" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 35609 } } peers { permanent_uuid: "0b5e0121090f4ea88ebee32253a84632" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 36363 } }
I20260808 08:29:41.105386 29231 sys_catalog.cc:455] T 00000000000000000000000000000000 P af48b9c76e7340da879a969e0b91f6f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af48b9c76e7340da879a969e0b91f6f0" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 35979 } } peers { permanent_uuid: "7195aacdff92472d9b183fcdf0baaead" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 35609 } } peers { permanent_uuid: "0b5e0121090f4ea88ebee32253a84632" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 36363 } } }
I20260808 08:29:41.105654 29231 sys_catalog.cc:458] T 00000000000000000000000000000000 P af48b9c76e7340da879a969e0b91f6f0 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 08:29:41.106144 28935 sys_catalog.cc:564] T 00000000000000000000000000000000 P af48b9c76e7340da879a969e0b91f6f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:29:41.106235 28897 raft_consensus.cc:482] T 00000000000000000000000000000000 P af48b9c76e7340da879a969e0b91f6f0 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 08:29:41.106344 28897 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af48b9c76e7340da879a969e0b91f6f0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:29:41.107553 28897 raft_consensus.cc:504] T 00000000000000000000000000000000 P af48b9c76e7340da879a969e0b91f6f0 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af48b9c76e7340da879a969e0b91f6f0" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 35979 } } peers { permanent_uuid: "7195aacdff92472d9b183fcdf0baaead" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 35609 } } peers { permanent_uuid: "0b5e0121090f4ea88ebee32253a84632" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 36363 } }
I20260808 08:29:41.107995 29197 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b5e0121090f4ea88ebee32253a84632 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af48b9c76e7340da879a969e0b91f6f0" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 35979 } } peers { permanent_uuid: "7195aacdff92472d9b183fcdf0baaead" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 35609 } } peers { permanent_uuid: "0b5e0121090f4ea88ebee32253a84632" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 36363 } }
I20260808 08:29:41.108220 29197 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b5e0121090f4ea88ebee32253a84632 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b5e0121090f4ea88ebee32253a84632, State: Initialized, Role: FOLLOWER
I20260808 08:29:41.108544 29197 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b5e0121090f4ea88ebee32253a84632 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af48b9c76e7340da879a969e0b91f6f0" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 35979 } } peers { permanent_uuid: "7195aacdff92472d9b183fcdf0baaead" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 35609 } } peers { permanent_uuid: "0b5e0121090f4ea88ebee32253a84632" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 36363 } }
I20260808 08:29:41.109582 29016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "af48b9c76e7340da879a969e0b91f6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7195aacdff92472d9b183fcdf0baaead"
I20260808 08:29:41.109792 29016 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7195aacdff92472d9b183fcdf0baaead [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:29:41.109824 29197 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b5e0121090f4ea88ebee32253a84632 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 08:29:41.111083 29016 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7195aacdff92472d9b183fcdf0baaead [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af48b9c76e7340da879a969e0b91f6f0 in term 1.
I20260808 08:29:41.111559 29155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "af48b9c76e7340da879a969e0b91f6f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "0b5e0121090f4ea88ebee32253a84632"
I20260808 08:29:41.111778 29155 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b5e0121090f4ea88ebee32253a84632 [term 0 FOLLOWER]: Advancing to term 1
I20260808 08:29:41.108510 28897 leader_election.cc:288] T 00000000000000000000000000000000 P af48b9c76e7340da879a969e0b91f6f0 [CANDIDATE]: Term 1 election: Requested vote from peers 7195aacdff92472d9b183fcdf0baaead (127.22.235.253:35609), 0b5e0121090f4ea88ebee32253a84632 (127.22.235.252:36363)
I20260808 08:29:41.113305 28857 leader_election.cc:302] T 00000000000000000000000000000000 P af48b9c76e7340da879a969e0b91f6f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7195aacdff92472d9b183fcdf0baaead, af48b9c76e7340da879a969e0b91f6f0; no voters:
W20260808 08:29:41.112084 29248 catalog_manager.cc:1558] T 00000000000000000000000000000000 P af48b9c76e7340da879a969e0b91f6f0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 08:29:41.113453 29248 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 08:29:41.113727 29231 raft_consensus.cc:2781] T 00000000000000000000000000000000 P af48b9c76e7340da879a969e0b91f6f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 08:29:41.113940 29231 raft_consensus.cc:686] T 00000000000000000000000000000000 P af48b9c76e7340da879a969e0b91f6f0 [term 1 LEADER]: Becoming Leader. State: Replica: af48b9c76e7340da879a969e0b91f6f0, State: Running, Role: LEADER
I20260808 08:29:41.113062 29155 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0b5e0121090f4ea88ebee32253a84632 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af48b9c76e7340da879a969e0b91f6f0 in term 1.
I20260808 08:29:41.114336 29231 consensus_queue.cc:237] T 00000000000000000000000000000000 P af48b9c76e7340da879a969e0b91f6f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af48b9c76e7340da879a969e0b91f6f0" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 35979 } } peers { permanent_uuid: "7195aacdff92472d9b183fcdf0baaead" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 35609 } } peers { permanent_uuid: "0b5e0121090f4ea88ebee32253a84632" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 36363 } }
I20260808 08:29:41.116124 29231 sys_catalog.cc:455] T 00000000000000000000000000000000 P af48b9c76e7340da879a969e0b91f6f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af48b9c76e7340da879a969e0b91f6f0. Latest consensus state: current_term: 1 leader_uuid: "af48b9c76e7340da879a969e0b91f6f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af48b9c76e7340da879a969e0b91f6f0" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 35979 } } peers { permanent_uuid: "7195aacdff92472d9b183fcdf0baaead" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 35609 } } peers { permanent_uuid: "0b5e0121090f4ea88ebee32253a84632" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 36363 } } }
I20260808 08:29:41.116238 29231 sys_catalog.cc:458] T 00000000000000000000000000000000 P af48b9c76e7340da879a969e0b91f6f0 [sys.catalog]: This master's current role is: LEADER
W20260808 08:29:41.123119 29273 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0b5e0121090f4ea88ebee32253a84632: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 08:29:41.123212 29273 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 08:29:41.123528 29249 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b5e0121090f4ea88ebee32253a84632 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af48b9c76e7340da879a969e0b91f6f0" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 35979 } } peers { permanent_uuid: "7195aacdff92472d9b183fcdf0baaead" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 35609 } } peers { permanent_uuid: "0b5e0121090f4ea88ebee32253a84632" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 36363 } } }
I20260808 08:29:41.123638 29249 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b5e0121090f4ea88ebee32253a84632 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 08:29:41.126438 29287 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 08:29:41.128996 29287 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 08:29:41.145038 28942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:41.125055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45456 (local address 127.22.235.254:35979)
0808 08:29:41.125185 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:41.125190 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:41.128879 (+ 3689us) server_negotiation.cc:407] Connection header received
0808 08:29:41.129071 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:41.129074 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:41.129121 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:41.129169 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:41.131411 (+ 2242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:41.132202 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:41.136717 (+ 4515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:41.137064 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:41.137137 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:41.141641 (+ 4504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:41.141645 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:41.141648 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:41.141678 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:41.143132 (+ 1454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:41.143642 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:41.143646 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:41.143648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:41.143740 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:41.144031 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:41.144034 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:41.144036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:41.144247 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:41.144366 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:41.144740 (+ 374us) server_negotiation.cc:299] Negotiation successful
0808 08:29:41.144870 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 08:29:41.145680 29077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:41.125535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45792 (local address 127.22.235.253:35609)
0808 08:29:41.125645 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:41.125649 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:41.130322 (+ 4673us) server_negotiation.cc:407] Connection header received
0808 08:29:41.130422 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:41.130425 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:41.130477 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:41.130560 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:41.131653 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:41.132440 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:41.133659 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:41.134025 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:41.134098 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:41.139358 (+ 5260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:41.139362 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:41.139365 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:41.139394 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:41.141052 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:41.143940 (+ 2888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:41.143944 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:41.143946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:41.144048 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:41.144331 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:41.144334 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:41.144336 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:41.144540 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:41.144663 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:41.145407 (+ 744us) server_negotiation.cc:299] Negotiation successful
0808 08:29:41.145532 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 08:29:41.146729 29155 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0b5e0121090f4ea88ebee32253a84632 [term 1 FOLLOWER]: Refusing update from remote peer af48b9c76e7340da879a969e0b91f6f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:29:41.147876 29016 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7195aacdff92472d9b183fcdf0baaead [term 1 FOLLOWER]: Refusing update from remote peer af48b9c76e7340da879a969e0b91f6f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 08:29:41.148132 29208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:41.125932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40268 (local address 127.22.235.252:36363)
0808 08:29:41.126032 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:41.126036 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:41.131314 (+ 5278us) server_negotiation.cc:407] Connection header received
0808 08:29:41.132325 (+ 1011us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:41.132328 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:41.132377 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:41.132445 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:41.133389 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:41.134132 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:41.137118 (+ 2986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:41.137471 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:41.137543 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:41.141478 (+ 3935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 08:29:41.141482 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 08:29:41.141484 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 08:29:41.141511 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 08:29:41.143330 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:41.144965 (+ 1635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:41.144969 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:41.144971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:41.145060 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 08:29:41.147133 (+ 2073us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 08:29:41.147136 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 08:29:41.147138 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 08:29:41.147357 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 08:29:41.147475 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:41.147867 (+ 392us) server_negotiation.cc:299] Negotiation successful
0808 08:29:41.147984 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 08:29:41.148582 29274 consensus_queue.cc:1035] T 00000000000000000000000000000000 P af48b9c76e7340da879a969e0b91f6f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7195aacdff92472d9b183fcdf0baaead" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 35609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:29:41.149309 29274 consensus_queue.cc:1035] T 00000000000000000000000000000000 P af48b9c76e7340da879a969e0b91f6f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b5e0121090f4ea88ebee32253a84632" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 36363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 08:29:41.153115 29211 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7195aacdff92472d9b183fcdf0baaead [sys.catalog]: SysCatalogTable state changed. Reason: New leader af48b9c76e7340da879a969e0b91f6f0. Latest consensus state: current_term: 1 leader_uuid: "af48b9c76e7340da879a969e0b91f6f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af48b9c76e7340da879a969e0b91f6f0" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 35979 } } peers { permanent_uuid: "7195aacdff92472d9b183fcdf0baaead" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 35609 } } peers { permanent_uuid: "0b5e0121090f4ea88ebee32253a84632" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 36363 } } }
I20260808 08:29:41.153131 29249 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b5e0121090f4ea88ebee32253a84632 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af48b9c76e7340da879a969e0b91f6f0. Latest consensus state: current_term: 1 leader_uuid: "af48b9c76e7340da879a969e0b91f6f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af48b9c76e7340da879a969e0b91f6f0" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 35979 } } peers { permanent_uuid: "7195aacdff92472d9b183fcdf0baaead" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 35609 } } peers { permanent_uuid: "0b5e0121090f4ea88ebee32253a84632" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 36363 } } }
I20260808 08:29:41.153244 29249 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b5e0121090f4ea88ebee32253a84632 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 08:29:41.153245 29211 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7195aacdff92472d9b183fcdf0baaead [sys.catalog]: This master's current role is: FOLLOWER
I20260808 08:29:41.163556 29274 sys_catalog.cc:455] T 00000000000000000000000000000000 P af48b9c76e7340da879a969e0b91f6f0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "af48b9c76e7340da879a969e0b91f6f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af48b9c76e7340da879a969e0b91f6f0" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 35979 } } peers { permanent_uuid: "7195aacdff92472d9b183fcdf0baaead" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 35609 } } peers { permanent_uuid: "0b5e0121090f4ea88ebee32253a84632" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 36363 } } }
I20260808 08:29:41.163715 29274 sys_catalog.cc:458] T 00000000000000000000000000000000 P af48b9c76e7340da879a969e0b91f6f0 [sys.catalog]: This master's current role is: LEADER
I20260808 08:29:41.163833 29296 mvcc.cc:205] Tried to move back new op lower bound from 7316184191560335360 to 7316184191447748608. Current Snapshot: MvccSnapshot[applied={T|T < 7316184191560335360}]
I20260808 08:29:41.165128 29249 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b5e0121090f4ea88ebee32253a84632 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af48b9c76e7340da879a969e0b91f6f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af48b9c76e7340da879a969e0b91f6f0" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 35979 } } peers { permanent_uuid: "7195aacdff92472d9b183fcdf0baaead" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 35609 } } peers { permanent_uuid: "0b5e0121090f4ea88ebee32253a84632" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 36363 } } }
I20260808 08:29:41.165251 29249 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b5e0121090f4ea88ebee32253a84632 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 08:29:41.165639 29298 mvcc.cc:205] Tried to move back new op lower bound from 7316184191560335360 to 7316184191447748608. Current Snapshot: MvccSnapshot[applied={T|T < 7316184191560335360}]
I20260808 08:29:41.166244 29231 sys_catalog.cc:455] T 00000000000000000000000000000000 P af48b9c76e7340da879a969e0b91f6f0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "af48b9c76e7340da879a969e0b91f6f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af48b9c76e7340da879a969e0b91f6f0" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 35979 } } peers { permanent_uuid: "7195aacdff92472d9b183fcdf0baaead" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 35609 } } peers { permanent_uuid: "0b5e0121090f4ea88ebee32253a84632" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 36363 } } }
I20260808 08:29:41.166358 29231 sys_catalog.cc:458] T 00000000000000000000000000000000 P af48b9c76e7340da879a969e0b91f6f0 [sys.catalog]: This master's current role is: LEADER
I20260808 08:29:41.167474 29297 mvcc.cc:205] Tried to move back new op lower bound from 7316184191560335360 to 7316184191447748608. Current Snapshot: MvccSnapshot[applied={T|T < 7316184191560335360}]
I20260808 08:29:41.167785 29211 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7195aacdff92472d9b183fcdf0baaead [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af48b9c76e7340da879a969e0b91f6f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af48b9c76e7340da879a969e0b91f6f0" member_type: VOTER last_known_addr { host: "127.22.235.254" port: 35979 } } peers { permanent_uuid: "7195aacdff92472d9b183fcdf0baaead" member_type: VOTER last_known_addr { host: "127.22.235.253" port: 35609 } } peers { permanent_uuid: "0b5e0121090f4ea88ebee32253a84632" member_type: VOTER last_known_addr { host: "127.22.235.252" port: 36363 } } }
I20260808 08:29:41.167897 29211 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7195aacdff92472d9b183fcdf0baaead [sys.catalog]: This master's current role is: FOLLOWER
I20260808 08:29:41.177624 29287 catalog_manager.cc:1347] Generated new cluster ID: 316f3c9644f845bc801f9f5536cf3a7d
I20260808 08:29:41.177706 29287 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 08:29:41.191640 29287 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 08:29:41.192936 29287 catalog_manager.cc:1504] Loading token signing keys...
I20260808 08:29:41.206485 29287 catalog_manager.cc:5868] T 00000000000000000000000000000000 P af48b9c76e7340da879a969e0b91f6f0: Generated new TSK 0
I20260808 08:29:41.207417 29287 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 08:29:41.238056 28942 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:29:41.234878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45468 (local address 127.22.235.254:35979)
0808 08:29:41.235022 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:41.235027 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:41.236703 (+ 1676us) server_negotiation.cc:407] Connection header received
0808 08:29:41.237151 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:41.237156 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:41.237214 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:41.237270 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:41.237902 (+ 632us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45468: BlockingRecv error: recv got EOF from 127.0.0.1:45468 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260808 08:29:41.239156 29077 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:29:41.235631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45806 (local address 127.22.235.253:35609)
0808 08:29:41.235742 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:41.235746 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:41.237062 (+ 1316us) server_negotiation.cc:407] Connection header received
0808 08:29:41.238410 (+ 1348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:41.238413 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:41.238464 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:41.238513 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:41.239043 (+ 530us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45806: BlockingRecv error: recv got EOF from 127.0.0.1:45806 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260808 08:29:41.240744 29208 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:29:41.236345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40280 (local address 127.22.235.252:36363)
0808 08:29:41.236477 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:41.236481 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:41.239647 (+ 3166us) server_negotiation.cc:407] Connection header received
0808 08:29:41.240015 (+ 368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:41.240019 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:41.240071 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:41.240121 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:41.240607 (+ 486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40280: BlockingRecv error: recv got EOF from 127.0.0.1:40280 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260808 08:29:41.245163 29077 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:29:41.243635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45820 (local address 127.22.235.253:35609)
0808 08:29:41.243731 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:41.243734 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:41.244180 (+ 446us) server_negotiation.cc:407] Connection header received
0808 08:29:41.244387 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:41.244390 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:41.244462 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:41.244515 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:41.245069 (+ 554us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45820: BlockingRecv error: recv got EOF from 127.0.0.1:45820 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260808 08:29:41.270191 29077 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:29:41.261513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45830 (local address 127.22.235.253:35609)
0808 08:29:41.261644 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:41.261648 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:41.268974 (+ 7326us) server_negotiation.cc:407] Connection header received
0808 08:29:41.269247 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:41.269251 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:41.269335 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:41.269401 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:41.270060 (+ 659us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45830: BlockingRecv error: recv got EOF from 127.0.0.1:45830 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260808 08:29:41.289956 29077 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:29:41.285123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45832 (local address 127.22.235.253:35609)
0808 08:29:41.285256 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:41.285260 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:41.288931 (+ 3671us) server_negotiation.cc:407] Connection header received
0808 08:29:41.289142 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:41.289146 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:41.289201 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:41.289253 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:41.289829 (+ 576us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45832: BlockingRecv error: recv got EOF from 127.0.0.1:45832 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260808 08:29:41.298314 29077 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:29:41.296518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45844 (local address 127.22.235.253:35609)
0808 08:29:41.296683 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:41.296687 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:41.297256 (+ 569us) server_negotiation.cc:407] Connection header received
0808 08:29:41.297477 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:41.297480 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:41.297553 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:41.297606 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:41.298187 (+ 581us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45844: BlockingRecv error: recv got EOF from 127.0.0.1:45844 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260808 08:29:41.309746 29077 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:29:41.308579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45858 (local address 127.22.235.253:35609)
0808 08:29:41.308732 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:41.308736 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:41.308946 (+ 210us) server_negotiation.cc:407] Connection header received
0808 08:29:41.309089 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:41.309092 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:41.309143 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:41.309211 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:41.309631 (+ 420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45858: BlockingRecv error: recv got EOF from 127.0.0.1:45858 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
W20260808 08:29:41.328768 29077 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:29:41.327656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45870 (local address 127.22.235.253:35609)
0808 08:29:41.327794 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:41.327799 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:41.327937 (+ 138us) server_negotiation.cc:407] Connection header received
0808 08:29:41.328071 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:41.328075 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:41.328130 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:41.328233 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:41.328645 (+ 412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45870: BlockingRecv error: recv got EOF from 127.0.0.1:45870 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260808 08:29:41.363883 29077 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:29:41.362718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45884 (local address 127.22.235.253:35609)
0808 08:29:41.362889 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:41.362894 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:41.362988 (+ 94us) server_negotiation.cc:407] Connection header received
0808 08:29:41.363207 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:41.363210 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:41.363272 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:41.363343 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:41.363762 (+ 419us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45884: BlockingRecv error: recv got EOF from 127.0.0.1:45884 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260808 08:29:41.431221 29077 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:29:41.430012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45886 (local address 127.22.235.253:35609)
0808 08:29:41.430173 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:41.430179 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:41.430301 (+ 122us) server_negotiation.cc:407] Connection header received
0808 08:29:41.430472 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:41.430475 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:41.430531 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:41.430605 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:41.431094 (+ 489us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45886: BlockingRecv error: recv got EOF from 127.0.0.1:45886 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260808 08:29:41.562269 29077 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:29:41.561240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45892 (local address 127.22.235.253:35609)
0808 08:29:41.561426 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:41.561430 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:41.561525 (+ 95us) server_negotiation.cc:407] Connection header received
0808 08:29:41.561646 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:41.561649 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:41.561700 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:41.561771 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:41.562144 (+ 373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45892: BlockingRecv error: recv got EOF from 127.0.0.1:45892 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
W20260808 08:29:41.821647 29077 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 08:29:41.820551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45906 (local address 127.22.235.253:35609)
0808 08:29:41.820699 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:41.820703 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:41.820736 (+ 33us) server_negotiation.cc:407] Connection header received
0808 08:29:41.820885 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:41.820888 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:41.820937 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:41.821014 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 08:29:41.821539 (+ 525us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45906: BlockingRecv error: recv got EOF from 127.0.0.1:45906 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 08:29:42.102557 29225 catalog_manager.cc:1259] Loaded cluster ID: 316f3c9644f845bc801f9f5536cf3a7d
I20260808 08:29:42.102646 29225 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7195aacdff92472d9b183fcdf0baaead: loading cluster ID for follower catalog manager: success
I20260808 08:29:42.104305 29225 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7195aacdff92472d9b183fcdf0baaead: acquiring CA information for follower catalog manager: success
I20260808 08:29:42.105546 29225 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7195aacdff92472d9b183fcdf0baaead: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 08:29:42.124456 29273 catalog_manager.cc:1259] Loaded cluster ID: 316f3c9644f845bc801f9f5536cf3a7d
I20260808 08:29:42.124557 29273 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0b5e0121090f4ea88ebee32253a84632: loading cluster ID for follower catalog manager: success
I20260808 08:29:42.126242 29273 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0b5e0121090f4ea88ebee32253a84632: acquiring CA information for follower catalog manager: success
I20260808 08:29:42.127403 29273 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0b5e0121090f4ea88ebee32253a84632: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 08:29:42.339361 29412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:42.335801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45916 (local address 127.22.235.253:35609)
0808 08:29:42.336071 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:42.336075 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:42.336093 (+ 18us) server_negotiation.cc:407] Connection header received
0808 08:29:42.336234 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:42.336236 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:42.336368 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:42.336461 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:29:42.336675 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:42.337182 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:42.337973 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:42.338276 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:42.338354 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:42.338620 (+ 266us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:29:42.339045 (+ 425us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 08:29:42.339135 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:42.339161 (+ 26us) server_negotiation.cc:299] Negotiation successful
0808 08:29:42.339208 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":89,"threads_started":1}
I20260808 08:29:42.345359 29422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:42.342280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40290 (local address 127.22.235.252:36363)
0808 08:29:42.342482 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:42.342486 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:42.342500 (+ 14us) server_negotiation.cc:407] Connection header received
0808 08:29:42.342601 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:42.342604 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:42.342711 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:42.342800 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:29:42.342929 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:42.343416 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:42.344121 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:42.344323 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:42.344387 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:42.344598 (+ 211us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:29:42.345030 (+ 432us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 08:29:42.345116 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:42.345166 (+ 50us) server_negotiation.cc:299] Negotiation successful
0808 08:29:42.345217 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":63,"threads_started":1}
I20260808 08:29:42.351806 29412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:42.348265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45920 (local address 127.22.235.253:35609)
0808 08:29:42.348364 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:42.348368 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:42.348812 (+ 444us) server_negotiation.cc:407] Connection header received
0808 08:29:42.348974 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:42.348978 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:42.349029 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:42.349102 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:29:42.349314 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:42.349968 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:42.350761 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:42.351003 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:42.351096 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:42.351335 (+ 239us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:29:42.351414 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 08:29:42.351490 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:42.351626 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 08:29:42.351674 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 08:29:42.352036 29422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:42.348059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40302 (local address 127.22.235.252:36363)
0808 08:29:42.348208 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:42.348212 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:42.348421 (+ 209us) server_negotiation.cc:407] Connection header received
0808 08:29:42.348591 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:42.348594 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:42.348665 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:42.348729 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:29:42.349156 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:42.349676 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:42.350960 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:42.351225 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:42.351293 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:42.351573 (+ 280us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:29:42.351649 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 08:29:42.351711 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:42.351870 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 08:29:42.351922 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 08:29:42.352649 29434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 08:29:42.347980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45472 (local address 127.22.235.254:35979)
0808 08:29:42.348253 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0808 08:29:42.348257 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 08:29:42.348296 (+ 39us) server_negotiation.cc:407] Connection header received
0808 08:29:42.348465 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 08:29:42.348469 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 08:29:42.348603 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 08:29:42.348743 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 08:29:42.348942 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:42.349692 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:42.350938 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 08:29:42.351253 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 08:29:42.351344 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 08:29:42.351743 (+ 399us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 08:29:42.352191 (+ 448us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 08:29:42.352271 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 08:29:42.352423 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 08:29:42.352482 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":77,"threads_started":1}
I20260808 08:29:42.355419 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 28825
I20260808 08:29:42.370136 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 28948
I20260808 08:29:42.381259 23471 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGDnrCb/build/debug/bin/kudu with pid 29080
2026-08-08T08:29:42Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3246 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6443 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (289174 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST